How Cloudflare Proxy Silently Broke My Lambda ALB Communication

Source: Dev.to

The Flow (before fix)

flowchart LR
    Browser -->|①| CF1[Cloudflare Edge (api.hoge.com)]
    CF1 -->|②| Lambda
    Lambda -->|③ backend.hoge.com = Cloudflare IP| CF2[Cloudflare Edge (backend.hoge.com)]
    CF2 -->|❌ Error 1000| ALB

Quick fix

Turn Proxy OFF for backend.hoge.com.
Long‑term plan: move Lambda → ALB communication inside the VPC.

The Error

Calling the API from the frontend returned 403 Forbidden with the body:

Cloudflare Error 1000
DNS points to prohibited IP

API Gateway and Lambda appeared healthy; there were no logs on the ECS side, so the ALB/WAF were not the culprits.

Architecture

flowchart LR
    Browser -->|HTTPS| APIGW[API Gateway]
    APIGW --> Lambda
    Lambda -->|HTTPS backend.hoge.com| ALB
    ALB --> ECS
    ECS --> RDS

Lambda acts as a BFF (Backend‑for‑Frontend). The backend runs on ALB + ECS (a legacy constraint). Lambda calls the ALB over HTTPS using the backend.hoge.com domain.

Troubleshooting Steps

1. Initial suspects – WAF rules, security‑group restrictions, ECS auth logic, API‑Gateway authorizer.

2. No ECS logs – indicated the request never reached the ALB.

3. curl test

   curl -v https://backend.hoge.com

   Response headers included:

   server: cloudflare

   Body contained DNS points to prohibited IP, confirming Cloudflare itself was returning the 403.

4. Docs check – Cloudflare Error 1000 occurs when an A record points to a Cloudflare‑owned IP or when a request is routed through another reverse proxy and ends up back at Cloudflare.

Root Cause

Cloudflare DNS settings:

• Because api.hoge.com had Proxy ON, browser traffic already passed through Cloudflare Edge before reaching Lambda.
• Lambda’s request to backend.hoge.com (also Proxy ON) resolved to a Cloudflare Anycast IP, sending the request back into Cloudflare Edge → loop.

Loop Diagram

flowchart LR
    Browser -->|①| CF1[Cloudflare Edge (api.hoge.com)]
    CF1 -->|②| Lambda
    Lambda -->|③ backend.hoge.com = Cloudflare IP| CF2[Cloudflare Edge (backend.hoge.com)]
    CF2 -->|❌ Error 1000| ALB
    ALB --> ECS
    ECS --> RDS

Why Error 1000?

Cloudflare returns Error 1000 when it detects a loop or when the resolved origin IP belongs to:

• Cloudflare‑owned IP ranges (loop prevention)
• RFC 1918 private addresses (10.x.x.x, 172.16.x.x, 192.168.x.x)
• Loopback address (127.0.0.1)

In our case, backend.hoge.com resolved to a Cloudflare IP, so Cloudflare treated the request as targeting itself and blocked it.

Fix #1 – Quick Fix

Turn Proxy OFF for backend.hoge.com.

Now DNS returns the actual origin CNAME (the ALB domain) instead of a Cloudflare IP, breaking the loop.

Post‑fix Flow

flowchart LR
    Lambda -->|backend.hoge.com = ALB domain| ALB
    ALB --> ECS
    ECS --> RDS

Requests flow normally again.

What I Learned

Cloudflare is more than DNS

It combines authoritative DNS, reverse‑proxy, CDN, and WAF. When Proxy ON, all traffic is routed through Cloudflare Edge before reaching the origin.

• Great for browser‑facing traffic (DDoS protection, caching, WAF).
• Potentially problematic for server‑to‑server communication if the origin is also behind Cloudflare.

Proxy ON vs OFF changes the entire traffic path

flowchart LR
    subgraph Proxy_OFF
        C1[Client] -->|ALB domain| ALB1[ALB]
    end
    subgraph Proxy_ON
        C2[Client] --> CF[Cloudflare Edge] --> ALB2[ALB]
    end

Match Proxy settings to your use case

Bottom line: Always verify Cloudflare proxy settings for any domain that will be called from within your own infrastructure. A simple “Proxy ON” can unintentionally create a request loop and surface as Cloudflare Error 1000.

Quick Fix (Temporary)

If a server calls a Proxy ON domain while the calling service is itself behind Cloudflare, you risk creating a loop that triggers Error 1000.

Long‑term Fix

Routing Lambda → ALB traffic through a public DNS name works, but it isn’t ideal.
Moving this communication inside the VPC is the proper solution.

flowchart LR
    Browser -->|HTTPS| APIGW[API Gateway]
    APIGW --> Lambda
    subgraph VPC
        Lambda -->|VPC‑internal| ALB
        ALB --> ECS
        ECS --> RDS
    end

Things to keep in mind for this migration

• Lambda must be deployed inside the VPC (if it isn’t already).
• Cold‑start impact from VPC placement is minimal today—AWS has significantly improved it.
• Security groups need an explicit rule allowing Lambda → ALB traffic.

This approach eliminates the Cloudflare dependency for internal traffic, reduces latency, and simplifies the network topology.

Why Did This Break Suddenly?

The Proxy ON configuration had been stable for a long time, so why did it start failing now?

• No official Cloudflare announcement about changes to Error 1000 detection or proxy behavior was found in the changelog.
• Community reports of “Error 1000 suddenly appearing” usually trace back to a user‑side configuration change (e.g., DNS record update, IP change on the hosting side).
• I’m still reviewing Cloudflare’s audit logs for anything that might explain the recent failures. I’ll update this post if I discover more details.

Summary

• When a request passes through a Proxy ON domain and that request subsequently calls another Proxy ON domain, Cloudflare creates a loop, resulting in Error 1000.
• For server‑to‑server communication, either turn Proxy OFF for internal domains or keep the traffic entirely inside the VPC.