[Paper] Mining Type Constructs Using Patterns in AI-Generated Code

Source: arXiv - 2602.17955v1

Overview

The paper presents the first large‑scale empirical study that compares how AI code generators (e.g., GitHub Copilot‑style agents) and human developers use TypeScript’s type system. By mining thousands of pull requests (PRs) from open‑source projects, the authors reveal striking differences in the frequency of risky type constructs—especially the over‑use of any—and show that, despite these safety concerns, AI‑generated PRs are accepted almost twice as often as human‑written ones.

Key Contributions

• Empirical dataset: Collected and annotated over 10 k TypeScript PRs from both AI‑assisted and human‑only contributions.
• Quantitative comparison: Measured the prevalence of type constructs (any, union types, type assertions, etc.) and identified a 9× higher usage of any in AI‑generated code.
• Advanced type misuse: Showed that AI agents more frequently employ “type‑bypassing” constructs (e.g., as unknown as T, // @ts-ignore).
• Acceptance paradox: Despite poorer type safety, AI‑authored PRs achieve a 1.8× higher acceptance rate than human PRs in the same repositories.
• Guidelines for practitioners: Provides concrete recommendations for developers who integrate AI assistants into their TypeScript workflow.

Methodology

1. Data collection – Scraped public GitHub repositories that use TypeScript and filtered PRs that were either (a) generated by known AI assistants (identified via commit metadata, bot accounts, or tool‑specific signatures) or (b) authored by human contributors.
2. Static analysis pipeline – Parsed each changed file with the TypeScript compiler API to extract every type‑related token (any, unknown, type assertions, generics, etc.).
3. Pattern mining – Used regular‑expression and AST‑based pattern matching to count occurrences of “risky” constructs (e.g., any, as any, // @ts-ignore).
4. Statistical testing – Evaluated differences between AI and human groups with chi‑square tests and effect‑size calculations to ensure statistical significance.
5. Outcome measurement – Treated PR acceptance (merged vs. closed without merge) as the primary success metric, complemented by review‑comment analysis to gauge reviewer concerns about type safety.

Results & Findings

• any overload: AI‑generated changes contain any in ≈ 12 % of modified lines versus ≈ 1.3 % for human edits.
• Type‑bypass constructs: Patterns such as as unknown as T appear 3.4× more often in AI PRs.
• Advanced types: AI agents do use sophisticated generics and conditional types, but often combine them with unsafe casts, diluting the benefit.
• Higher acceptance: AI PRs are merged 1.8× more frequently, suggesting reviewers prioritize functional correctness or speed of delivery over strict type safety.
• Reviewer feedback: When type‑related comments appear, they are more common in AI PRs, but many are ignored or resolved with quick fixes (e.g., adding a // @ts-ignore).

Practical Implications

• Code review tooling: Augment CI pipelines with stricter TypeScript linting (e.g., no-unsafe-any, no-explicit-any) to catch AI‑induced safety regressions before merge.
• AI assistant configuration: Prompt engineering can steer models toward safer patterns (e.g., “avoid any; prefer generics”). Some vendors already expose “type‑safety” knobs that can be toggled.
• Developer workflow: Treat AI‑generated snippets as drafts—run tsc --noEmit and static analysis locally before opening a PR.
• Hiring & onboarding: New developers can leverage AI for boilerplate code, but must be trained to audit the resulting types, especially in large codebases where any can silently propagate bugs.
• Productivity vs. safety trade‑off: The higher acceptance rate hints that AI can accelerate feature delivery, but organizations need policies to balance speed with long‑term maintainability.

Limitations & Future Work

• Scope to TypeScript: Findings may not generalize to languages with different type systems (e.g., Rust, Java).
• Bot identification: Some AI‑generated contributions could be mis‑tagged, affecting dataset purity.
• Acceptance bias: Projects that already use AI assistants might have relaxed type‑safety standards, inflating acceptance rates.
• Future directions: Extend the analysis to other languages, explore the impact of model size (e.g., GPT‑4 vs. Codex), and build automated “type‑safety auditors” that intervene during AI code generation.

Authors

• Imgyeong Lee
• Tayyib Ul Hassan
• Abram Hindle

Paper Information

• arXiv ID: 2602.17955v1
• Categories: cs.SE
• Published: February 20, 2026
• PDF: Download PDF