Building a Production-Ready Data Marketplace: Architecture, Security, and Lessons Learned

Source: Dev.to

The Problem Space

Building a marketplace seems straightforward until you start coding:

• Payments: Integrating Stripe, handling webhooks, managing checkout sessions
• Security: Encrypting API keys, managing sessions, preventing attacks
• Concurrency: Preventing overselling when multiple buyers compete for the last unit
• Access Control: Issuing tokens, managing permissions, handling revocation
• Testing: Ensuring everything works reliably in production

Each of these is a mini‑project in itself. UDAM solves all of them.

Architecture Overview

UDAM follows a classic three‑tier architecture, but with specific design choices optimized for marketplace needs:

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│   Frontend  │─────▶│   Backend    │─────▶│  PostgreSQL │
│  (Next.js)  │      │  (Node.js)   │      │   Database  │
└─────────────┘      └──────────────┘      └─────────────┘
                            │
                            ▼
                     ┌──────────────┐
                     │    Stripe    │
                     │   Payments   │
                     └──────────────┘

Tech Stack Choices

Backend: Node.js + Express

• Fast iteration for marketplace logic
• Excellent Stripe SDK support
• Large ecosystem for future extensions

Database: PostgreSQL

• ACID transactions (critical for payments)
• Row‑level locking (prevents race conditions)
• Mature, battle‑tested in production

Frontend: Next.js

• SSR capability for SEO (marketplace discoverability)
• Intentionally minimal – easy to customize
• API‑first design

Deep Dive: Token Encryption

The Challenge

Sellers provide API keys that buyers need to access their services. These keys must be:

• Encrypted at rest (database compromise shouldn’t expose keys)
• Decryptable for legitimate buyers (they need the actual key)
• Never logged or cached in plaintext

The Solution: AES‑256‑GCM

const crypto = require('crypto');

function encryptToken(apiKey, masterKey) {
  const iv = crypto.randomBytes(16);
  const cipher = crypto.createCipheriv('aes-256-gcm', masterKey, iv);

  let encrypted = cipher.update(apiKey, 'utf8', 'hex');
  encrypted += cipher.final('hex');

  const authTag = cipher.getAuthTag();

  return {
    encrypted,
    iv: iv.toString('hex'),
    authTag: authTag.toString('hex')
  };
}

Key Points

• GCM mode provides both encryption and authentication.
• Random IV ensures each encryption uses a unique initialization vector.
• Auth tag detects tampering attempts.
• Master key is stored securely in environment variables (never in code).

This approach means even if someone gains database access, they cannot decrypt the API keys without the master key.

Concurrency Control: The Overselling Problem

Scenario

The Solution: Row‑Level Locking

BEGIN;

-- Lock the row for this transaction
SELECT * FROM listings 
WHERE id = $1 
FOR UPDATE;

-- Check availability
IF available_units >= units_requested THEN
  UPDATE listings 
  SET available_units = available_units - $2
  WHERE id = $1;

  INSERT INTO orders (...) VALUES (...);
END IF;

COMMIT;

How it works

• FOR UPDATE locks the selected row until the transaction completes.
• Other transactions wait for the lock to be released.
• Only one transaction can decrement units at a time, making overselling impossible.

We verified this under load with a CI test that spawns 5 concurrent purchase attempts for 3 available units – exactly 3 orders succeed, 2 fail with “insufficient units”.

Payment Flow: Instant vs. Stripe Checkout

Instant Token Issuance (Small Orders)

For orders under a configurable threshold (e.g., $5):

• Order is created.
• Tokens are issued immediately.
• No payment confirmation needed.

Why? For small amounts, the friction of Stripe checkout hurts conversion more than the risk of fraud.

Stripe Checkout (Large Orders)

1. Create a Stripe Checkout session.
2. Redirect the user to Stripe.
3. Webhook confirms payment.
4. Tokens are issued after confirmation.

Implementation

if (totalPrice < SMALL_ORDER_LIMIT) {
  // Issue token instantly
  issueToken(orderId);
} else {
  // Create Stripe Checkout session
  const session = await stripe.checkout.sessions.create({
    payment_method_types: ['card'],
    line_items: [{price: priceId, quantity: 1}],
    mode: 'payment',
    success_url: `${BASE_URL}/success?session_id={CHECKOUT_SESSION_ID}`,
    cancel_url: `${BASE_URL}/cancel`
  });
  res.json({url: session.url});
}

Middleware Example (Auth)

async function authMiddleware(req, res, next) {
  const token = req.headers.authorization?.split(' ')[1];
  const session = await db.query(
    'SELECT * FROM sessions WHERE token = $1 AND expires_at > NOW()',
    [token]
  );

  if (!session.rows[0]) {
    return res.status(401).json({ error: 'Unauthorized' });
  }

  req.userId = session.rows[0].user_id;
  next();
}

Testing: CI/CD for Critical Flows

- name: E2E small-limit flow
  run: |
    TOKEN=$(curl -X POST /auth/login ...)
    LISTING_ID=$(curl -X POST /listings ...)
    ORDER=$(curl -X POST /orders ...)
    TOKENS=$(curl /tokens ...)

What we test

• ✅ Full purchase flow (login → create listing → buy → get tokens)
• ✅ Session revocation (logout → can’t access protected routes)
• ✅ Concurrency (5 simultaneous purchases for 3 units)
• ✅ Payment webhooks (in dev mode)

Performance Considerations

Database Indexes

CREATE INDEX idx_listings_status ON listings(status);
CREATE INDEX idx_orders_buyer ON orders(buyer_id);
CREATE INDEX idx_tokens_buyer ON tokens(buyer_id);
-- Additional indexes as needed for query patterns