Building Auth Validation: 5 Lessons on Making Error Messages Actually Helpful

Source: Dev.to

Why you should care

You’re building a tool that connects to multiple external services—GitHub, AI agents, various APIs. Everything works fine… until it doesn’t.

A user runs a job. It fails. They check the logs, scroll through output, and 30 minutes later discover: “Oh, the auth token expired.”

I built an auth validation feature for DevLoop Runner (a tool that automates PRs using AI agents). Here’s what I learned about making validation actually useful.

Lesson 1: Hide what users don’t need to see

The first implementation printed JWT tokens, email addresses, and organization names in plain text in the validation output—bad idea.

const isJsonString = authJsonPath.startsWith('{') || authJsonPath.startsWith('[');
const displayValue = isJsonString 
  ? '[JSON content - masked for security]' 
  : authJsonPath;

Users only need to know “Is it configured?” and “Does it work?”. Mask sensitive credentials.

Lesson 2: Validate what actually gets used

The validation command was checking an environment variable, while the execution command used a file. Validation passed, but execution failed, confusing users.

const homeDir = process.env.HOME || os.homedir();
const authFilePath = path.join(homeDir, '.codex', 'auth.json');

if (fs.existsSync(authFilePath)) {
  // Validate the file, not the env var
}

Validate the environment exactly as the runtime does.

Lesson 3: “Failed” doesn’t always mean failed

OpenAI and Anthropic API keys are optional in my tool. The first version reported status: failed when they weren’t configured, which was misleading. I changed it to status: skipped.

if (isBlank(apiKey)) {
  checks.push({
    name: 'OPENAI_API_KEY',
    status: 'skipped',
    message: 'Not configured (optional for follow‑up issue generation)',
  });
  return { status: 'passed', checks };
}

• Missing required config → failed
• Missing optional config → skipped

The distinction matters.

Lesson 4: Don’t ask for more than you need

GitHub token validation originally required three scopes: repo, workflow, read:org. The codebase never used read:org; only workflow was needed for GitHub Actions.

const REQUIRED_GITHUB_SCOPES = ['repo'];

Overly strict validation produces unnecessary error messages, frustrating users.

Lesson 5: Make your validation tool debuggable

A connectivity check timed out in Jenkins with no logs, making the issue hard to diagnose. I added verbose logging and increased the timeout.

await codexClient.executeTask({
  prompt: 'ping',
  maxTurns: 1,
  verbose: true,  // Now we can see what's happening
});

// Increased timeout from 10 s to 30 s for Docker environments
setTimeout(() => reject(new Error('Timeout after 30 seconds')), 30000);

If your validation tool is hard to debug, you’ve created another problem.

The pattern

Building validation isn’t just about technical correctness; it’s about the message you send to users:

• Mask what they don’t need to see
• Check what actually gets executed
• Distinguish between failed and skipped
• Don’t over‑require permissions
• Make failures debuggable

Validation results are communication. They should help users understand what to do next, not confuse them with technically correct but practically misleading messages.

More thoughts on building tools and making decisions at tielec.blog.