AI Trading: Lesson Learned #105: Post-Trade RAG Sync Was Missing

Source: Dev.to

Incident Summary

• Date: January 7, 2026
• Severity: HIGH
• Category: data-integrity, observability, mandate‑violation

Issue

During the CEO review on Jan 6, 2026 we discovered that trades were not being recorded to Vertex AI RAG or ChromaDB after execution.

• A pre‑session RAG check existed (reads lessons before trading).
• No post‑session sync was implemented (writes trades after trading).

Impact

• Violated the CLAUDE.md mandate:

  “Record every trade and lesson in BOTH ChromaDB AND Vertex AI RAG (MANDATORY)”

• JSON backup was functioning, but the data was not synced to the vector databases.
• CEO could not query trades via Dialogflow because the data wasn’t in Vertex AI.
• Trades from Jan 3‑6, 2026 were missing from the RAG databases.
• Dialogflow queries for trade history returned no results.
• The learning loop was broken – we could not learn from trades that weren’t recorded.

Root Cause

The post‑execution step that should sync trades to RAG was never implemented, leaving only the pre‑flight check in place.

Resolution

• Added scripts/sync_trades_to_rag.py to sync trades to both Vertex AI and ChromaDB.
• Updated daily-trading.yml with a “Sync Trades to RAG (Post‑Execution)” step that runs after every trade session, with a graceful fallback to the JSON backup.

Implementation Details

• Script: scripts/sync_trades_to_rag.py – handles writing to Vertex AI and ChromaDB.

• Workflow change:

  # daily-trading.yml
  - name: Sync Trades to RAG (Post-Execution)
    run: python scripts/sync_trades_to_rag.py
    continue-on-error: true   # fallback to JSON if sync fails

• The step executes after each trade session, ensuring both pre‑flight and post‑flight data persistence.

Recommendations

1. Verify both pre‑flight and post‑flight steps exist for any critical flow.
2. Test end‑to‑end data flow:
   • Execute trade → record to JSON → sync to RAG → query via Dialogflow.
3. Add monitoring for RAG corpus size vs. JSON trade count to catch future mismatches early.

Tags

rag, vertex-ai, chromadb, data-sync, post-execution, mandate-compliance

Related

More lessons: rag_knowledge/lessons_learned