[Paper] How Do Software Engineering Students Use Generative AI in Real-World Capstone Projects? An Empirical Baseline Study

Source: arXiv - 2604.24521v1

Overview

This study investigates how undergraduate software‑engineering students actually employ generative AI (GenAI) tools—such as large language models and code‑generation assistants—while delivering real‑world capstone projects for external clients. By surveying 150 participants across 18 teams, the authors establish a baseline of current practices, attitudes, and client expectations, offering concrete guidance for educators and industry mentors who are grappling with the rapid diffusion of AI‑assisted development.

Key Contributions

• Empirical baseline of GenAI usage across the full software‑engineering lifecycle in a large, authentic capstone setting (178 students, 15 client projects).
• Taxonomy of emerging workflows, distinguishing how teams integrate AI for requirements, design, coding, testing, and documentation.
• Student‑driven responsible‑use recommendations, emphasizing verification, code ownership, and maintaining independent problem‑solving skills.
• Client stakeholder perspective, revealing strong support for AI assistance but firm expectations around quality, data privacy, and developer understanding.
• Actionable pedagogical insights, including the need for explicit AI‑use policies, targeted AI‑literacy resources, and designated team governance roles.

Methodology

The researchers adopted a mixed‑methods approach:

1. Course Context – An undergraduate capstone module lasting four months, where AI tool usage was explicitly allowed.
2. Data Collection –
   • Quantitative survey (n = 150) covering attitudes, frequency of AI use, specific tasks, perceived benefits, and risks.
   • Qualitative open‑ended responses to capture nuanced workflows and concerns.
   • Client survey (project sponsors) to gauge expectations and apprehensions about student AI use.
3. Analysis – Descriptive statistics for usage prevalence, thematic coding for workflow patterns, and cross‑validation with client feedback to triangulate findings.

The design purposefully mirrors real‑world development environments, avoiding artificial constraints that could skew behavior.

Results & Findings

• Widespread Adoption: Over 80 % of students reported using GenAI at least once per week; the most common tools were GitHub Copilot, ChatGPT, and domain‑specific code generators.
• Lifecycle Coverage: AI was applied not only for code generation (≈ 70 % of teams) but also for drafting requirements, creating UML diagrams, writing test cases, and producing documentation.
• Emerging Workflows: Two dominant patterns emerged:
  1. Prompt‑first, verify‑later – Students generate code snippets via prompts and then manually test/review.
  2. Iterative refinement – Teams use AI to prototype, then iteratively refine with human feedback.
• Perceived Benefits: Faster prototyping, reduced boilerplate effort, and improved access to up‑to‑date APIs.
• Perceived Risks: Over‑reliance leading to shallow understanding, potential plagiarism, and inadvertent exposure of client data.
• Student Recommendations: Mandatory verification steps, documentation of AI‑generated artifacts, and periodic “knowledge‑check” sessions to ensure independent competence.
• Client Views: 90 % expressed enthusiasm for AI‑accelerated delivery, but 75 % demanded explicit assurances that students understand the code they submit and that no confidential data is fed into external models.

Practical Implications

• For Educators: Introduce clear, enforceable AI‑use policies (e.g., “AI‑assist, but you must own the final code”) and embed AI‑literacy modules that teach prompt engineering, result verification, and ethical considerations.
• For Industry Mentors: Leverage the same AI‑assisted workflows when onboarding junior developers, using the study’s taxonomy to set expectations around verification and knowledge retention.
• Tool Builders: Opportunity to design “enterprise‑safe” GenAI interfaces that allow on‑premise model execution or data‑scrubbing to satisfy client privacy concerns.
• Team Governance: Assign a “AI‑champion” role within each team to monitor usage, maintain logs of AI prompts, and ensure compliance with quality standards—mirroring code‑review practices.
• Hiring & Assessment: Recruiters can probe candidates on their ability to validate AI‑generated code, turning a potential risk into a measurable competency.

Limitations & Future Work

• Self‑Reported Data – Reliance on student surveys may under‑ or over‑estimate actual AI usage; direct instrumentation of IDEs could provide more precise metrics.
• Single Institution & Course – Findings reflect one university’s capstone structure; replication across diverse curricula and cultural contexts is needed.
• Tool Landscape Evolution – Rapid advances in GenAI mean that specific tool popularity may shift; longitudinal studies are required to track changing practices.
• Future Directions – The authors propose controlled experiments comparing guided vs. unguided AI use, development of automated provenance tracking for AI‑generated artifacts, and deeper exploration of how AI influences collaborative dynamics within teams.

Authors

• Michael Mircea
• Elisa Schmid
• Jakob Droste
• Kurt Schneider

Paper Information

• arXiv ID: 2604.24521v1
• Categories: cs.SE
• Published: April 27, 2026
• PDF: Download PDF