The False Positive Problem: Calibrating Crisis Detection Without Becoming The Boy Who Cried Wolf

Source: Dev.to

Part of the CrisisCore Build Log – trauma‑informed systems engineering

When I built crisis detection into Pain Tracker, I knew the stakes were high in both directions:

• Miss a real crisis → Someone doesn’t get help when they need it.
• Trigger on normal behavior → The system becomes annoying noise, users disable it, and then you’ve lost them when it actually matters.

This is the calibration problem. Here’s how I approached it.

The Core Challenge: Fast Clicks ≠ Panic

The first version of my crisis detection was embarrassingly naive:

// 🚫 Don't do this
if (clicksPerSecond > 5) {
  activateEmergencyMode(); // 😬
}

Within hours I had reports of people accidentally triggering emergency mode while:

• Scrolling through body‑map locations
• Quickly rating multiple symptoms
• Just using the app normally on their phone

Rapid input isn’t distress; it’s often efficiency.

Multi‑Signal Detection: The Weighted Approach

The fix was to stop looking for any single indicator and instead track a weighted constellation of signals:

// Calculate overall stress level from multiple factors
const overallStress =
  (currentIndicators.painLevel / 10) * 0.3 +               // User‑reported pain
  currentIndicators.cognitiveLoad * 0.25 +                // Task difficulty signals
  currentIndicators.inputErraticBehavior * 0.2 +          // Click pattern variance
  currentIndicators.errorRate * 0.15 +                    // Mistakes and corrections
  currentIndicators.frustrationMarkers * 0.1;             // Back navigation, help requests

// Only escalate when composite score crosses threshold
let severity = 'none';
if (overallStress >= 0.8) severity = 'critical';
else if (overallStress >= 0.6) severity = 'severe';
else if (overallStress >= 0.4) severity = 'moderate';
else if (overallStress >= 0.2) severity = 'mild';

No single signal can trigger emergency mode on its own. You need convergent evidence.

What Actually Counts as “Erratic Input”?

Fast clicking doesn’t matter. Irregular clicking does.

Someone efficiently tapping through a familiar flow will have consistent intervals between clicks. Someone struggling will show high variance—long pauses followed by frustrated rapid taps.

const calculateInputErraticBehavior = useCallback(() => {
  if (clickTimes.current.length  Date.now() - time  a + b, 0) / intervals.length;
  const variance = intervals.reduce(
    (sum, interval) => sum + Math.pow(interval - avgInterval, 2),
    0
  ) / intervals.length;

  return Math.min(1, variance / 10000); // Normalize to 0‑1
});

• High variance + high error rate + elevated pain level → Something’s probably wrong.
• High click rate + consistent intervals + normal error rate → Power user; leave them alone.

The Frustration Stack

Beyond click patterns, we track what I call the frustration stack:

interface CrisisTrigger {
  type: 'pain_spike' | 'cognitive_fog' | 'rapid_input' |
        'error_pattern' | 'emotional_distress' | 'timeout';
  value: number;
  threshold: number;
  timestamp: Date;
  context: string;
}

Each trigger has its own threshold, tuned by a sensitivity setting:

Users choose their sensitivity. Some want the system watching closely; others find it intrusive. Both are valid.

Cognitive Load ≠ Crisis

Distinguishing “I’m working through something complex” from “I’m drowning” is tricky. High cognitive load alone isn’t a problem; it becomes a signal when combined with rising error rate, help requests, and back‑navigation.

const calculateCognitiveLoad = useCallback(() => {
  const recentErrors = errorEvents.current.filter(
    time => Date.now() - time.getTime()  Date.now() - time.getTime()  {
  if (crisisSettings.autoActivation.enabled) {
    // … activation logic …
  } else if (crisisLevel === 'none' && isCrisisModeActive) {
    // Auto‑deactivate when stress returns to normal
    setTimeout(() => {
      setIsCrisisModeActive(false);
      setCrisisFeatures(prev => ({
        ...prev,
        emergencyMode: false,
        cognitiveFogSupport: false,
        multiModalInput: false,
      }));
    }, 5000); // 5‑second delay to prevent flapping
  }
}, [crisisLevel, crisisSettings.autoActivation, isCrisisModeActive]);

Why the delay? Crisis states aren’t binary switches. Someone might calm down briefly, then spike again. Rapid mode‑switching is disorienting and erodes trust. The 5‑second buffer creates hysteresis—the system requires sustained stability before standing down.

Recovery Flows: Graceful Exit from Emergency Mode

Emergency mode activation is immediate; deactivation is gradual.

When a user manually resolves a crisis or the system detects sustained calm:

const deactivateEmergencyMode = useCallback(() => {
  setIsCrisisModeActive(false);
  setCrisisFeatures({
    emergencyMode: false,
    cognitiveFogSupport: false,
    multiModalInput: false,
    stressResponsiveUI: true, // This stays on
  });
  resetCrisisDetection('resolved');
}, [resetCrisisDetection]);

What stays active: stress‑responsive UI. Even after crisis resolution, we keep monitoring. The guardrails stay up longer than the emergency sirens.

Session Recording: Learning From Each Episode

Every crisis episode becomes a learning opportunity—for the user and for the system:

interface CrisisSession {
  id: string;
  startTime: Date;
  endTime?: Date;
  triggers: CrisisTrigger[];
  responses: CrisisResponse[];
  userActions: string[];
  outcome: 'resolved' | 'escalated' | 'timed_out' | 'user_dismissed' | 'ongoing';
  duration: number;
  effectiveInterventions: string[];
  userFeedback?: string;
}

This data is stored locally for the user’s own pattern recognition. Over time, they might notice:

• “My crisis episodes usually start with back‑navigation loops.”
• “Simplified mode actually helps me finish entries.”
• “I tend to dismiss too early.”

Self‑knowledge is the ultimate calibration.

The Sensitivity Dial: User Control

Different people need different trigger points. The system offers three sensitivity levels:

const thresholds = {
  painLevel:
    preferences.crisisDetectionSensitivity === 'high' ? 7 :
    preferences.crisisDetectionSensitivity === 'medium' ? 8 : 9,
  // … other thresholds follow the same pattern …
};

Allowing users to select low, medium, or high sensitivity lets them balance safety and intrusiveness according to personal preference.