AI Engineering: Advent of AI with goose Day 11
Source: Dev.to
Day 11: Photo Booth AI Application 📸
The Challenge: Build a Real‑Time Filter App in One Day
- Open on your phone
- See yourself with fun filters
- Filters track your face
- Switch between effects
- Capture the photo
- Download it
- Share it
The Fun House Photo Booth (Built with Subagents)
I split the work into specialized subagents, mimicking a real dev team:
Subagent 1 – Core App Builder
- Built the HTML/CSS/JS structure
- Implemented camera access and live video preview
- Added capture + download functionality
- Made the UI mobile‑responsive
Subagent 2 – Filter Engineer
- Integrated MediaPipe Face Landmarker (468‑point face mesh)
- Built the real‑time filter system anchored to specific landmarks
- Implemented filter switching
Optional Subagents I Added
- Stylist – polished the UI (FilterSense branding)
- Documentation Writer – created usage notes
- Performance Optimizer – ensured smooth tracking
Subagents let me parallelize the work while keeping the build clean and modular.
Tech Stack
- goose Subagents – task orchestration
- Sonnet 4.5 by Anthropic
- HTML/CSS/JS – core app
- MediaPipe Face Landmarker – on‑device spatial intelligence
- Canvas API – rendering filters + mesh
- SessionStorage – storing captured images
- QR workflow – sharing
- Mobile‑first responsive layout
No backend; everything runs locally in the browser.
My Experience (From Camera to AR Filters)
- Select a filter
- Watch it attach to your face
- Move, tilt, smile – it follows
- Capture the moment
- Download or share
The result feels like a lightweight AR app running directly in the browser.
What My Application Does
- Opens the camera instantly
- Tracks the face using MediaPipe
- Renders a 468‑point mesh
- Applies filters anchored to landmarks
- Allows users to switch filters
- Captures a clean photo
- Stores it safely (SessionStorage)
- Redirects to an export page
- Supports download + QR sharing
- Works smoothly on mobile
Spatial Intelligence (MediaPipe Face Landmarker)
One of the most advanced parts of this build is the on‑device spatial intelligence. The entire face‑tracking pipeline runs in the browser, eliminating the need for server‑side processing.
Why this matters
- Real‑time performance
- Low latency
- Offline capability
- Privacy‑preserving
- No external compute required
How it works
- 468 face landmarks with 3D positional data
- Stable tracking across movement
- Mesh topology that can be removed at any time
These landmarks drive the entire filter system.
Mesh Rendering
- Glowing neon nodes
- Connecting edges
- Animated mesh movement
This visualizes the underlying AI in real time.
Filter Anchoring
{
"Crown": { "landmark": 10, "offsetY": -60 },
"Beard": { "landmark": 152, "offsetY": 40 },
"Reindeer Eyelashes": { "landmark": 159, "offsetY": -10 }
}The coordinates ensure perfect alignment as the user moves.
Clean Capture Pipeline
- Create a fresh canvas
- Draw the video frame
- Draw only the mesh (no external PNGs)
- Export as PNG
- Store in
sessionStorage - Redirect to export page
This guarantees consistent captures across browsers.
Technical Highlights
- On‑device spatial intelligence enables real‑time AR effects without external compute.
- Landmark‑driven positioning gives sub‑pixel accuracy for filter anchoring.
- Fully responsive UI with a capture, download, and QR‑based sharing workflow.
Insights
- Subagents feel like a real dev team, even when you’re the sole contributor.
- MediaPipe’s local inference is incredibly powerful.
- Clean capture pipelines matter for cross‑browser consistency.
- Spatial intelligence unlocks AR‑level experiences.
- Declarative workflows scale beautifully.
- Mobile‑first design is essential for real‑world use.
Powered By
- goose by Block (powered by Sonnet 4.5 by Anthropic)
- MediaPipe by Google
- HTML/CSS/JS (my own design + engineering workflow)
My Final Thoughts
Day 11: Solved Filter Sense Photo Booth – delivered. Festival magic: activated.
This post is part of my Advent of AI journey “AI Engineering: Advent of AI with goose Day 11.”