From Algorithms to Agents: How My Research in Clustering Shapes My Automation Logic

Source: Dev.to

Key Insights

• Research Shapes Thinking – Academic principles influence how I approach automation problems.
• Noise = Flakiness – The same mental model for filtering spatial noise applies to test stability.
• Efficiency Matters – Algorithmic thinking drives optimization in wait strategies and element selection.
• Pattern Recognition – The mindset of “finding order in chaos” applies to self‑healing frameworks.

“Just click this, type that, check this.”

If this is how you think about test automation, you’re building a house of cards.

Most automation engineers focus on actions—what to click, what to type, what to assert. Treating automation as a mere sequence of actions leads to brittle scripts that shatter the moment the UI changes a single class name.

I don’t look at automation as scripting. I look at it as a data problem.

Why? Because long before I was building automation frameworks in Python, my co‑author Hrishav and I were researching algorithmic efficiency in spatial databases. That research—published in 2012—didn’t teach me a specific technique to copy‑paste into Selenium, but it fundamentally shaped how I think about complex data problems.

The Foundation: The TDCT Algorithm

In 2012, Hrishav and I co‑authored a research paper titled “A Density Based Clustering Technique For Large Spatial Data Using Polygon Approach” (TDCT).

The Problem We Solved

How do you find meaningful patterns (clusters) in massive, chaotic datasets—without getting overwhelmed by noise?

Existing algorithms like DBSCAN were good, but they struggled with:

• Arbitrary shapes – Real‑world data doesn’t form neat circles.
• Computational cost – Scanning every point against every other point doesn’t scale.
• Noise sensitivity – Outliers distorted the cluster boundaries.

Our Solution: Triangular Density

Instead of using circular neighborhoods (as DBSCAN does), we mapped data points into triangular polygons. This allowed us to:

• Calculate density more efficiently than radial scans.
• Detect clusters of arbitrary, non‑convex shapes.
• Isolate noise points without corrupting core clusters.

Key Insight

By changing the geometry of the problem (circles → triangles), we reduced computational complexity while improving cluster detection accuracy. This collaborative work laid the foundation for how I approach complex data problems to this day.

The Bridge: Why This Matters for Quality Engineering

“Dhiraj, what does spatial clustering have to do with Selenium?”

Not the code—the mindset.

It’s About Problem Framing

When I encounter a complex automation challenge, I don’t immediately think “what Selenium command do I need?” I ask:

• What’s the data structure here? (The DOM is a tree; test results are time‑series data.)
• What’s the noise vs. the signal? (Which element attributes are stable? Which failures are true bugs?)
• How can I reduce complexity? (Can I optimize the problem’s “geometry” like TDCT did?)

This mental model—trained by years of algorithmic research—influences every framework decision I make.

Applying the Mindset: Practical Examples

Example 1: Multi‑Attribute Element Location with Fallback Logic

Brute‑Force Approach (single point of failure):

# If ID changes, everything breaks
element = driver.find_element(By.ID, "checkout-btn-v3")

Algorithmic Approach (cluster multiple attributes by reliability):

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException

def find_element_with_fallback(driver, strategies: list[tuple]) -> WebElement:
    """
    Analyze multiple 'data points' (attributes) ordered by reliability.
    Returns the first element that becomes clickable.
    """
    for strategy, locator in strategies:
        try:
            element = WebDriverWait(driver, 2).until(
                EC.element_to_be_clickable((strategy, locator))
            )
            return element
        except TimeoutException:
            continue
    raise NoSuchElementException("All strategies exhausted")

checkout_strategies = [
    (By.CSS_SELECTOR, "[data-testid='checkout']"),               # Most stable: test IDs
    (By.CSS_SELECTOR, "button[aria-label='Checkout']"),         # Accessibility attrs
    (By.XPATH, "//button[contains(text(), 'Checkout')]"),        # Text content
    (By.CSS_SELECTOR, ".cart-section button.primary"),         # Structural fallback
]

checkout_btn = find_element_with_fallback(driver, checkout_strategies)

The first strategy acts as the “density core.” If it fails, we gracefully fall back to less stable but still valid “neighbors.”

Example 2: Self‑Healing Element Location

Static Approach (brittle, noise‑sensitive):

driver.find_element(By.ID, "submit-btn-v3")  # Breaks when ID changes

Adaptive Approach (cluster‑like resilience):

def self_heal_find(driver, primary_locator, fallback_locators):
    """
    Try the primary locator; if it fails, evaluate fallback locators
    based on attribute stability and similarity to known anchors.
    """
    try:
        return driver.find_element(*primary_locator)
    except NoSuchElementException:
        for loc in fallback_locators:
            candidates = driver.find_elements(*loc)
            if candidates:
                # Simple heuristic: pick the element with the most stable attributes
                return max(candidates, key=lambda e: evaluate_stability(e))
    raise NoSuchElementException("Element could not be healed")

def evaluate_stability(element):
    # Placeholder for a real scoring function (e.g., based on data‑testids,
    # ARIA labels, relative position to stable anchors, etc.)
    score = 0
    if element.get_attribute("data-testid"):
        score += 3
    if element.get_attribute("aria-label"):
        score += 2
    # Add more heuristics as needed
    return score

primary = (By.ID, "submit-btn-v3")
fallbacks = [
    (By.CSS_SELECTOR, "[data-testid='submit']"),
    (By.XPATH, "//button[contains(text(),'Submit')]"),
    (By.CSS_SELECTOR, "button.primary")
]

submit_btn = self_heal_find(driver, primary, fallbacks)

The approach isn’t a literal implementation of TDCT, but the thinking is the same: evaluate multiple data points and select the most reliable combination.

Tools Reflecting This Philosophy

When I created packages like Lumos ShadowDOM or Visual Guard, I wasn’t consciously implementing clustering algorithms. Yet the design decisions echo the same principles:

• Traversing Shadow DOM efficiently → Understand the structure before brute‑forcing.
• Visual regression with SSIM → Use mathematical models (instead of pixel‑by‑pixel noise) to find meaningful differences.
• Self‑healing in frameworks → Treat element attributes as “data points” with varying reliability.

Research doesn’t give me copy‑paste solutions; it gives me a lens for seeing automation as a data problem, not a scripting problem.

Conclusion: Automation Isn’t Just Code—It’s Logic

Whether it’s the TDCT algorithm published years ago or the automation tools and libraries I build today, the goal remains the same:

Bringing order to chaos.

The DOM is chaotic. Test automation, when approached with a research‑driven mindset, can impose structure, reduce flakiness, and scale gracefully.