Multi-Tenant Design for Bedrock Knowledge Base: Solving the Account Limit with Metadata Filtering

Source: Dev.to

Introduction

Recently, while working with Bedrock KnowledgeBase in my daily work, I encountered some challenges related to its specifications that I’d like to share.

Background

Currently, I’m developing a multi-tenant application using Bedrock KnowledgeBase (KB). KB is an orchestrator for implementing LLM RAG that handles vectorization of files into vector stores and can generate context‑aware conversations when combined with Bedrock Agent.

We’re using OpenSearch as our vector store, and our design creates separate KBs and indices for each tenant. This approach ensures data isolation between tenants, which seemed like a natural design choice at the time.

The Problem

When checking Bedrock’s quotas, I found the Knowledge bases per account quota, which has a hard limit of 100. With our initial design, this meant the application could only support up to 100 tenants, so we needed to reconsider our architecture.

Solution

We modified the design so that KBs and indices are shared across multiple tenants. Because KBs have several parameters related to vectorization (e.g., ChunkStrategy), we created several combinations of ChunkStrategy and MaxToken and let users select from these options for sharing.

An important consideration is ensuring that tenant data isn’t referenced during conversations with other tenants. KB allows attaching custom metadata during vectorization, so we attach a tenant_id‑like metadata field and filter documents by that ID during retrieval.

KB metadata documentation

Architecture overview

• Shared Knowledge Base across multiple tenants
• Custom metadata (tenant_id) attached to each document
• Metadata filtering during retrieval to ensure data isolation

Attaching metadata to documents

# ingest_documents
response = client.ingest_knowledge_base_documents(
    knowledgeBaseId='string',
    dataSourceId='string',
    clientToken='string',
    documents=[
        {
            'metadata': {
                'type': "IN_LINE_ATTRIBUTE",
                'inlineAttributes': [
                    {
                        'key': 'tenant_id',
                        'value': {
                            'type': "STRING",
                            'stringValue': "$tenant_id",
                        }
                    },
                ]
            },
            'content': {
                # document content here
            }
        },
    ]
)

Filtering documents by metadata during agent invocation

# invoke_agent
response = boto3.client('bedrock-agent').invoke_agent(
    knowledgeBaseConfigurations=[
        {
            "knowledgeBaseId": "$vector_store_id",
            "description": "Knowledge base for document retrieval",
            "retrievalConfiguration": {
                "vectorSearchConfiguration": {
                    "filter": {
                        "equals": {
                            "key": "tenant_id",
                            "value": "$tenant_id"
                        }
                    }
                }
            },
        }
    ]
)

Future Plans

With the above implementation, we can now build the application while successfully avoiding the 100‑KB limit.

In multi‑tenant applications, it’s crucial to verify that one tenant cannot access another tenant’s data. I plan to create a monitoring mechanism that:

1. Creates multiple test tenants.
2. Inserts distinct documents into the shared vector store for each tenant.
3. Queries the agent with questions about other tenants’ documents.
4. Ensures no answers are returned.

Running this script regularly in staging environments will help detect any unintended data leakage. Monitoring system resources (CPU, memory) is important, but monitoring data isolation is equally critical.

Conclusion

These are the issues related to KB specifications and our countermeasures. This experience highlighted the importance of checking cloud service specifications before deciding on system design. I hope this article is helpful to you.