Build a Real-Time tldraw Whiteboard with Velt Comments inside ChatGPT🤯🔥

Source: Dev.to

What We’re Building

We’re building a collaborative whiteboard that you can control with ChatGPT. You can tell ChatGPT to add shapes, put sticky notes, or change layouts, and the board will update instantly. Your teammates can join the board, see changes as they happen, and leave comments directly on the canvas.

We’ll use TLDraw for the canvas, Velt for real‑time collaboration, and the Model Context Protocol (MCP) to connect everything to ChatGPT. By the end, you’ll have a functional whiteboard app that works inside ChatGPT and responds to natural language.

Understanding the Foundation

A GPT App consists of two parts:

1. Web widget – the UI rendered inside ChatGPT (our whiteboard canvas).
2. MCP server – a Node.js service that defines the tools ChatGPT can call (e.g., “add a rectangle”).

When you say “draw a rectangle,” ChatGPT reads the tool definition, calls your MCP server, and the server updates the canvas.

TLDraw

• Handles the canvas, drawing tools, shapes, text.
• Provides real‑time board sync via @tldraw/sync – everyone in the same room sees updates instantly.

Velt

• Provides collaboration features: comments, live cursors, presence indicators.
• Works through React components on the frontend and a REST API on the backend.

These two libraries build the UI; the MCP server connects the UI to ChatGPT.

The MCP Server

The MCP server is a Node.js app that defines tools ChatGPT can invoke. Each tool includes a name, description, and an inputSchema describing required parameters.

const tools = [
  {
    name: "add-item",
    description: "Add an item to the list",
    inputSchema: {
      type: "object",
      properties: {
        text: {
          type: "string",
          description: "The item text"
        },
        priority: {
          type: "string",
          enum: ["low", "medium", "high"],
          description: "Item priority"
        }
      },
      required: ["text"]
    }
  }
];

The description guides ChatGPT on when to use the tool, while inputSchema tells it what data to send.

Prerequisites

Setup

git clone https://github.com/Studio1HQ/velt-app-examples
cd velt-app-examples
pnpm install

cd syncboard_server
pnpm install
cd ..

Chrome 142+ users: Disable the local‑network‑access‑check flag.

1. Open chrome://flags/
2. Search for local-network-access-check
3. Set to Disabled and restart Chrome.

Building the Whiteboard

The project is split into a frontend (widget) and a backend (MCP server).

src/syncboard/          # Frontend whiteboard
├── syncboard.jsx       # Canvas and Velt components
├── mockUsers.js        # Test users (Bob & Alice)
└── index.jsx           # Entry point

syncboard_server/       # Backend MCP server
└── src/
    ├── server.ts       # Tool definitions
    └── velt/           # Comment handlers

Setting Up TLDraw

Edit src/syncboard/syncboard.jsx and add the basic TLDraw canvas:

import { Tldraw } from 'tldraw';
import { useSyncDemo } from '@tldraw/sync';
import 'tldraw/tldraw.css';

function SyncboardCanvas() {
  const store = useSyncDemo({
    roomId: import.meta.env.VITE_TLDRAW_ROOM_ID // e.g., "my-room-abc"
  });

  return <Tldraw store={store} />;
}

useSyncDemo creates a synced store tied to the specified roomId. All participants in that room share the same canvas state.

Adding Velt

Wrap the TLDraw canvas with Velt’s provider and add collaboration components.

// syncboard.jsx
import {
  VeltProvider,
  useVeltClient,
  VeltComments,
  VeltPresence,
  VeltCursor,
  VeltCommentTool,
  VeltSidebarButton
} from '@veltdev/react';
import { Tldraw } from 'tldraw';
import { useSyncDemo } from '@tldraw/sync';
import 'tldraw/tldraw.css';
import { useEffect, useState } from 'react';

function SyncboardCanvas() {
  const store = useSyncDemo({
    roomId: import.meta.env.VITE_TLDRAW_ROOM_ID
  });

  const { client } = useVeltClient();
  const [veltReady, setVeltReady] = useState(false);

  // Initialize Velt client
  useEffect(() => {
    const init = async () => {
      if (!client || veltReady) return;

      await client.identify(currentUser, { forceReset: true });
      await client.setDocument('syncboard-whiteboard', {
        documentName: 'Syncboard Collaborative Whiteboard'
      });

      setVeltReady(true);
    };
    init();
  }, [client, veltReady]);

  return (
    <>
      {/* Top bar with collaboration controls */}
      {veltReady && <VeltSidebarButton />}
      {veltReady && <VeltCommentTool />}

      {/* Main canvas */}
      <Tldraw store={store} />

      {/* Velt overlays */}
      {veltReady && (
        <>
          <VeltComments />
          <VeltPresence />
          <VeltCursor />
        </>
      )}
    </>
  );
}

export default function Syncboard() {
  return (
    <VeltProvider>
      <SyncboardCanvas />
    </VeltProvider>
  );
}

The Velt components add comment threads, presence indicators, and live cursors on top of the TLDraw canvas.

MCP Server (Backend)

Create the MCP server in syncboard_server/src/server.ts. Below is a minimal example that defines a tool for adding a rectangle to the board.

import { createMcpServer } from '@openai/mcp';
import { addRectangle } from './velt/rectangleHandler'; // your custom logic

const tools = [
  {
    name: "add-rectangle",
    description: "Add a rectangle shape to the whiteboard",
    inputSchema: {
      type: "object",
      properties: {
        width: { type: "number", description: "Width of the rectangle" },
        height: { type: "number", description: "Height of the rectangle" },
        color: { type: "string", description: "Fill color (hex)" }
      },
      required: ["width", "height"]
    }
  }
];

const server = createMcpServer({
  tools,
  handler: async (toolName, params) => {
    if (toolName === "add-rectangle") {
      await addRectangle(params);
    }
    // Add more tool handlers as needed
  }
});

server.listen(3000, () => console.log("MCP server listening on port 3000"));

addRectangle should interact with the Velt document (via its REST API or SDK) to insert a rectangle shape into the TLDraw store.

Expose the server with ngrok so ChatGPT can reach it:

ngrok http 3000

Copy the generated HTTPS URL and configure it in the ChatGPT app settings as the MCP endpoint.

Running the App

1. Start the frontend (widget) – usually via Vite or your preferred dev server.
2. Start the MCP server (node syncboard_server/src/server.ts).
3. Expose the server with ngrok.
4. In ChatGPT, add a new custom app and point it to the ngrok URL.
5. Interact! Try prompts like:
   • “Draw a blue rectangle 200 px wide and 100 px tall.”
   • “Add a sticky note that says ‘Sprint goals’.”

ChatGPT will call the appropriate tool, the MCP server will update the Velt document, and the changes will appear instantly on the shared whiteboard.