[Paper] DeCEAT: Decoding Carbon Emissions for AI-driven Software Testing

Source: arXiv - 2602.18012v1

Overview

The paper introduces DeCEAT, a framework that measures the carbon footprint and performance of small language models (SLMs) when they’re used to generate automated software tests. By combining energy‑tracking tools with classic test‑quality metrics, the authors show that sustainability isn’t a single number – it depends on model choice, prompt design, and the trade‑offs developers are willing to make.

Key Contributions

• First systematic sustainability audit for SLM‑driven test generation – moves the focus beyond the usual large‑model analyses.
• DeCEAT framework that couples CodeCarbon (energy & CO₂ tracking) with HumanEval‑based test coverage, all under reproducible hardware and runtime settings.
• Prompt‑variant study using the Anthropic template to demonstrate how subtle changes in prompt wording affect both emissions and test quality.
• Empirical trade‑off matrix that categorizes models by (i) low‑energy/fast, (ii) high‑stability, and (iii) high‑accuracy under carbon constraints.
• Open‑source tooling and a reproducible benchmark suite, enabling other teams to plug in their own models or prompts.

Methodology

1. Benchmark selection – The authors use the HumanEval suite (a collection of 164 Python coding problems) as a realistic proxy for test‑generation workloads.
2. Model roster – Several publicly available SLMs (e.g., GPT‑2‑small, LLaMA‑7B, Anthropic‑Claude‑mini) are evaluated under identical hardware (single‑GPU, fixed batch size).
3. Prompt engineering – Two prompt families are built from the Anthropic “template” style: a baseline prompt and three adaptive variants that tweak temperature, few‑shot examples, and instruction phrasing.
4. Energy tracking – CodeCarbon runs alongside each inference session, logging power draw, runtime, and estimated CO₂ emissions (based on regional electricity mix).
5. Quality assessment – Generated unit tests are run against the hidden solutions; coverage, pass‑rate, and flakiness are recorded.
6. Time‑aware analysis – Results are plotted on a three‑axis chart (energy, latency, coverage) to surface multidimensional sustainability profiles.

Results & Findings

• Energy vs. Accuracy trade‑off: Larger SLMs (LLaMA‑7B) achieve higher coverage but consume ~2× the energy of the smallest model.
• Prompt impact: Adaptive prompts consistently shave 10‑15 % off both energy and latency while keeping coverage within a few points of the baseline.
• Stability gains: Prompt tweaks that add clearer instruction boundaries reduce flaky test generation by up to 3 percentage points.
• Multidimensional sustainability: No single model dominates across all axes; developers must prioritize based on project constraints (e.g., CI budget vs. test thoroughness).

Practical Implications

• CI/CD cost budgeting: Teams can now estimate the carbon cost of adding AI‑generated tests to their pipelines and choose a model/prompt combo that fits a given carbon budget.
• Prompt‑first optimization: Instead of swapping out models, a quick prompt rewrite can deliver measurable energy savings—useful for rapid prototyping.
• Green‑by‑design testing tools: The DeCEAT framework can be integrated into test‑generation libraries (e.g., pytest-gen, codex-cli) to expose real‑time emission stats to developers.
• Policy & reporting: Organizations seeking ESG compliance can leverage the provided metrics to report AI‑induced emissions for software quality activities.
• Model selection guidance: For projects where test coverage is critical (e.g., safety‑critical code), a larger SLM may be justified; for routine codebases, a small SLM with adaptive prompts offers a low‑carbon alternative.

Limitations & Future Work

• Hardware specificity: Experiments were run on a single GPU type; results may shift on CPUs, TPUs, or newer accelerator generations.
• Scope of benchmarks: HumanEval covers Python functions; other languages or larger codebases could exhibit different energy‑quality dynamics.
• Carbon factor granularity: Emission estimates rely on average regional electricity mixes; real‑world data centers may have greener or dirtier grids.
• Prompt space exploration: Only a handful of adaptive prompts were tested; automated prompt‑search (e.g., reinforcement learning) could uncover even better sustainability profiles.
• Model ecosystem: The study focuses on a limited set of open SLMs; extending to proprietary models (e.g., OpenAI’s Codex) would broaden applicability.

DeCEAT opens the conversation about the hidden environmental cost of AI‑assisted testing and gives developers a concrete toolbox to make greener, data‑driven decisions.

Authors

• Pragati Kumari
• Novarun Deb

Paper Information

• arXiv ID: 2602.18012v1
• Categories: cs.SE
• Published: February 20, 2026
• PDF: Download PDF