Teaching AI Agents by Example: A New Paradigm in Automation

Source: Dev.to

Introduction

For decades, we’ve taught computers through explicit instructions: write code, define selectors, handle every edge case. But what if we could teach AI agents the same way we teach humans—by showing them what to do?

Traditional browser automation relies on brittle selectors and rigid scripts. When a website updates its CSS classes, your automation breaks. When a button moves, your script fails. You’re not describing what you want done—you’re prescribing exactly how to do it. This creates a maintenance nightmare: every UI change requires code updates, and every new workflow demands engineering time. Domain experts who know the work can’t automate it without developer help.

The Idea: Teaching by Demonstration

What if you could simply record yourself performing a task and have an AI extract the intent, workflow, and decision points? That’s the idea behind SkillForge—a tool that converts screen recordings into structured skill files that any AI agent can execute.

How SkillForge Works

1. Record – Capture yourself performing any web‑based task.
2. Extract – AI analyzes the recording to identify goals, workflows, and UI elements.
3. Generate – Output is a human‑readable SKILL.md file with context‑aware instructions.
4. Execute – Any compatible agent can perform the skill, adapting to UI changes.

Benefits

• Resilience – Skills survive UI updates because they describe intent, not selectors.
• Accessibility – Domain experts can create automation without writing code.
• Portability – Skills are usable across different agent frameworks.
• Transparency – The format is human‑readable and auditable.

Skill File Format

The generated SKILL.md follows a simple markdown structure:

• Goal – What the agent should accomplish.
• Workflow – Step‑by‑step execution plan.
• Context – How to identify elements semantically.
• Error handling – Recovery strategies.

Compatibility

The format works with AutoGen, LangChain, CrewAI, or any framework that can parse structured instructions.

Looking Ahead

We’re moving toward a world where AI agents can learn from observation just like humans do. Capturing domain expertise through demonstration rather than code opens up automation to everyone.

Links

• Product Hunt: https://www.producthunt.com/products/skillforge-2
• Website: https://skillforge.expert

---

What workflows would you teach an AI agent if you could just show it once?