[Paper] CodeR3: A GenAI-Powered Workflow Repair and Revival Ecosystem

Source: arXiv - 2511.19510v1

Overview

Scientific workflows—think of them as reproducible pipelines that glue together data, tools, and domain expertise—are aging faster than we’d like. The authors introduce CodeR³, a generative‑AI‑driven system that can “repair, revive, and reuse” legacy workflows (e.g., Taverna) by translating them into modern, actively‑maintained platforms such as Snakemake and VisFlow. The work shows how AI can cut down the tedious manual work required to keep old pipelines alive, while still keeping a human in the loop for the tricky bits.

Key Contributions

• AI‑powered workflow parsing: Uses large language models (LLMs) to understand the structure and intent of decayed Taverna workflows.
• Automated migration pipeline: Generates equivalent Snakemake/VisFlow scripts, handling syntax conversion, dependency mapping, and service discovery.
• Stepwise visual analysis: Provides an interactive visualization of each workflow stage, making it easy for users to spot where things went wrong.
• Service substitution engine: Suggests modern alternatives for obsolete web services or command‑line tools, ranked by relevance and community feedback.
• Human‑in‑the‑loop validation framework: Allows domain experts to approve or tweak AI‑generated replacements, ensuring scientific correctness.
• Crowdsourcing platform prototype: Enables the community to collectively revive, test, and certify legacy workflows, turning workflow decay into a collaborative maintenance effort.

Methodology

1. Workflow Ingestion: The system reads a Taverna workflow’s XML description, extracts nodes (services), data links, and metadata.
2. LLM‑based Semantic Extraction: A fine‑tuned generative model (e.g., GPT‑4) is prompted with the extracted snippets to infer the high‑level purpose of each node (e.g., “align reads”, “run statistical test”).
3. Mapping to Modern Primitives: The inferred semantics are matched against a curated registry of Snakemake rules and VisFlow components. When a direct match isn’t found, the model proposes a substitution (e.g., replace a retired SOAP service with a Dockerized CLI tool).
4. Code Generation: Using the same LLM, the system emits runnable Snakemake/VisFlow code, embedding appropriate conda environments or container specifications.
5. Visualization & Review: A web UI visualizes the original and generated pipelines side‑by‑side, highlighting nodes that required substitution. Domain experts can approve, edit, or reject suggestions.
6. Iterative Refinement: Approved changes are fed back to the model to improve future suggestions, and the final pipeline is executed on a test dataset to verify output consistency.

Results & Findings

• Parsing Accuracy: In a benchmark of 30 real‑world Taverna workflows, the AI correctly identified 92 % of service intents, dramatically reducing manual inspection time.
• Migration Success Rate: 24 out of 30 workflows (80 %) were fully translated into functional Snakemake scripts with only minor human adjustments.
• Effort Reduction: Average manual effort dropped from ~6 hours per workflow (baseline) to ~1.5 hours when using CodeR³, a 75 % time saving.
• Human Intervention Hotspots: Service substitution (especially for proprietary or discontinued APIs) and data format validation still required expert review in ~30 % of cases.
• Crowdsourced Validation: Early testing of the prototype crowdsourcing portal showed that community members could confirm 85 % of revived workflows within a week, indicating strong collaborative potential.

Practical Implications

• Extended Shelf‑life for Legacy Pipelines: Organizations can resurrect valuable, previously‑published analyses without rewriting them from scratch, preserving reproducibility.
• Accelerated Onboarding: New team members can quickly understand and adapt old workflows thanks to the visual stepwise analysis and modern code output.
• Reduced Technical Debt: By moving to container‑aware platforms (Snakemake, VisFlow), teams automatically gain benefits like reproducible environments and easier CI/CD integration.
• Community‑driven Maintenance: The crowdsourcing layer turns workflow decay into a shared responsibility, similar to open‑source bug triaging, fostering a healthier ecosystem for scientific software.
• Potential for Automation in Other Domains: The same AI‑driven parsing‑to‑translation pipeline could be repurposed for legacy ETL jobs, CI pipelines, or even infrastructure‑as‑code scripts.

Limitations & Future Work

• Domain‑Specific Knowledge Gaps: The LLM sometimes misinterprets highly specialized services, leading to incorrect substitutions that only a domain expert can catch.
• Service Discovery Database: The current registry of modern alternatives is manually curated; scaling it will require automated harvesting of tool metadata from repositories like BioContainers or Conda‑Forge.
• Validation Scalability: Running full end‑to‑end tests on large datasets can be costly; future work will explore lightweight provenance checks and synthetic test data generation.
• User Experience Studies: The paper presents early case studies; systematic usability testing with a broader developer audience is needed to refine the human‑in‑the‑loop UI.
• Extending Beyond Taverna: While the prototype focuses on Taverna, adapting the pipeline to other legacy workflow systems (e.g., Kepler, Pegasus) is a natural next step.

Authors

• Asif Zaman
• Kallol Naha
• Khalid Belhajjame
• Hasan M. Jamil