如何处理 AI Agent Cron 作业中的部分失败

Source: Dev.to

TL;DR

学习如何检测、恢复以及跟踪 AI‑agent 定时任务中的部分失败。此方法将我们的成功率从 70 % → 95 % 提升，并处理核心功能成功但次要操作失败的情况。

前置条件

• AI‑agent 框架（OpenClaw 或类似）
• 基于 Cron 的计划任务
• 外部 API 依赖（社交发布、消息投递）
• 警报渠道（Slack、Discord 等）

问题：隐藏的部分失败

典型的 AI‑agent 定时任务流程：

# x-poster-morning example
✅ Post to X API (200 OK)
❌ Message delivery fails (Timeout/Rate Limit)
→ What's the job status?

传统的二元方法

if curl -X POST "$API_ENDPOINT"; then
  echo "SUCCESS"
  exit 0
else
  echo "FAILED"
  exit 1
fi

此逻辑将“发布成功 + 投递失败”视为成功，这具有误导性。

步骤 1 – 细粒度状态跟踪

逐个跟踪每个子任务并得出整体状态。

#!/bin/bash
declare -A RESULTS
OVERALL_SUCCESS=true

# Step 1: Post to X
if post_to_x "${CONTENT}"; then
  RESULTS[post]="✅ SUCCESS"
else
  RESULTS[post]="❌ FAILED"
  OVERALL_SUCCESS=false
fi

# Step 2: Message delivery
if deliver_message "${RESULT_MSG}"; then
  RESULTS[delivery]="✅ SUCCESS"
else
  RESULTS[delivery]="⚠️ FAILED"
  # Not a complete failure since posting succeeded
fi

# Step 3: Overall status determination
if [[ "${RESULTS[post]}" == *"SUCCESS"* ]]; then
  STATUS="PARTIAL_SUCCESS"
  if [[ "${RESULTS[delivery]}" == *"SUCCESS"* ]]; then
    STATUS="FULL_SUCCESS"
  fi
else
  STATUS="FULL_FAILURE"
fi

步骤 2 – 分层 Slack 通知

根据派生的状态发送不同的消息。

report_to_slack() {
  local status=$1
  case $status in
    "FULL_SUCCESS")
      openclaw message send --channel slack --target 'C091G3PKHL2' \
        --message "✅ x-poster-morning: All operations successful"
      ;;
    "PARTIAL_SUCCESS")
      openclaw message send --channel slack --target 'C091G3PKHL2' \
        --message "⚠️ x-poster-morning: Core success, delivery failed
Core: ${RESULTS[post]}
Delivery: ${RESULTS[delivery]}
Manual review needed"
      ;;
    "FULL_FAILURE")
      openclaw message send --channel slack --target 'C091G3PKHL2' \
        --message "❌ x-poster-morning: Complete failure
${RESULTS[post]}
Immediate action required"
      ;;
  esac
}

第3步 – 智能重试逻辑

仅对失败的组件进行重试，使用指数退避。

retry_failed_delivery() {
  local max_attempts=3
  local attempt=1

  while [ $attempt -le $max_attempts ]; do
    echo "Delivery retry attempt $attempt/$max_attempts"

    if deliver_message "${CACHED_RESULT}"; then
      RESULTS[delivery]="✅ SUCCESS (retry $attempt)"
      return 0
    fi

    sleep $((attempt * 10))   # exponential back‑off
    ((attempt++))
  done

  RESULTS[delivery]="❌ FAILED after $max_attempts retries"
  return 1
}

第 4 步 – 状态持久化

存储部分失败状态，以便后续分析和手动恢复。

STATE_FILE="/Users/anicca/.openclaw/workspace/cron-state/${JOB_NAME}-$(date +%Y-%m-%d).json"

save_job_state() {
  cat > "$STATE_FILE"  "${METRICS_FILE}.tmp" && mv "${METRICS_FILE}.tmp" "$METRICS_FILE"
}

weekly_report() {
  echo "## Cron Success Rate (Last 7 days)"
  jq -r '
    to_entries |
    map(select(.key >= (now - 7*24*3600 | strftime("%Y-%m-%d")))) |
    map(.value | to_entries | map(.value.status)) |
    flatten |
    group_by(.) |
    map({status: .[0], count: length}) |
    .[]
  ' "$METRICS_FILE"
}

#!/bin/bash
set -euo pipefail

JOB_NAME="x-poster-morning"
declare -A RESULTS
RETRY_COUNT=0
STATE_FILE="/Users/anicca/.openclaw/workspace/cron-state/${JOB_NAME}-$(date +%Y-%m-%d).json"

main() {
  # Execute core logic
  execute_core_logic

  # Determine initial status
  determine_overall_status

  # Retry delivery if partial failure
  if [[ "$STATUS" == "PARTIAL_SUCCESS" ]]; then
    retry_failed_delivery
    determine_overall_status   # Re‑evaluate after retry
  fi

  # Persist state, notify, update metrics
  save_job_state
  report_to_slack "$STATUS"
  update_metrics

  # Exit codes for monitoring tools
  case "$STATUS" in
    "FULL_SUCCESS")   exit 0 ;;
    "PARTIAL_SUCCESS") exit 1 ;;   # Attention needed but not critical
    "FULL_FAILURE")   exit 2 ;;   # Critical
  esac
}

main "$@"

通过这些步骤，您可以可靠地检测、恢复并监控 AI 代理定时任务中的部分失败，将隐藏的错误转化为可操作的数据，显著提升整体可靠性。

关键要点

这种方法将我们的 AI 代理定时任务成功率从 70 % 提升至 95 %。恰当的部分失败处理提升了系统可靠性，并减少了人工干预的需求。