I Replaced My LangSmith MCP Server with a 200-Token CLI Skill

Source: Dev.to

If you’re using LangSmith with Claude Code (or any AI‑coding agent), you’re probably running the official MCP server. It works, but every session injects 5 000+ tokens of tool schemas into your context window — even if you never touch LangSmith.

I built langsmith‑cli to fix that. It’s a standalone CLI and a Claude Code plugin that replaces the MCP server with a —dataset my-eval-set.

Smart Filtering That Translates to FQL

Nobody wants to write raw Filter Query Language. The CLI translates human‑friendly flags automatically:

# These flags...
langsmith-cli runs list --tag summarizer --failed --last 24h --slow

# ...become this FQL:
# and(
#   has(tags, "summarizer"),
#   eq(error, true),
#   gt(start_time, "2026-03-03T..."),
#   gt(latency, "5s")
# )

Time presets like --recent (last hour), --today, --last 7d, and --since 2026-01-01 all work. Content search with --grep supports regex and field‑specific matching. Everything composes.

What’s New in v0.4.0

The v0.4.0 release focused on type safety and code quality:

• Zero pyright errors – every function has proper type annotations (client: langsmith.Client, not client: Any). Return types are real SDK Pydantic models, not object.
• datasets delete command with confirmation prompts and JSON‑mode support.
• Improved error handling across prompts and runs commands using specific SDK exception types (LangSmithNotFoundError, LangSmithConflictError) instead of a broad except Exception.
• 702 unit tests passing with real Pydantic model instances (no MagicMock for test data).

Getting Started

# Install
uv tool install langsmith-cli
# or:
pip install langsmith-cli

# Authenticate
export LANGSMITH_API_KEY="lsv2_..."
# or:
langsmith-cli auth login

# Start exploring
langsmith-cli runs list --project my-app --last 24h
langsmith-cli --json runs get-latest --failed --fields name,error

If you’re using Claude Code, add the plugin for the best agent experience:

claude plugin marketplace add gigaverse-app/langsmith-cli
claude plugin install langsmith-cli@langsmith-cli

The code is MIT licensed and on GitHub:
github.com/gigaverse-app/langsmith-cli

If you’re building with LangSmith and are tired of context‑heavy MCP servers, give it a try. Happy to hear feedback in the issues.