[Paper] Patterns of Bot Participation and Emotional Influence in Open-Source Development

Source: arXiv - 2601.11138v1

Overview

The paper investigates how automated bots participate in open‑source discussions within the Ethereum ecosystem and whether their presence nudges the emotional tone of human contributors. By analyzing nearly 37 k accounts across ten popular repositories, the authors show that even a tiny fraction of bots (≈0.3 %) can noticeably affect both the timing of conversations and the sentiment expressed by developers.

Key Contributions

• Empirical bot census: Identified and validated 105 bots out of 36 875 accounts, providing the first large‑scale quantification of bot prevalence in Ethereum‑related projects.
• Participation patterns: Discovered distinct temporal behaviors—bots are uniformly active on pull‑request (PR) threads, but tend to intervene later in issue discussions, while humans follow a U‑shaped activity curve.
• Speed advantage: Demonstrated that bots reply faster than humans on PRs, yet adopt a slower, “maintenance‑mode” cadence on issues.
• Emotion modeling: Trained a classifier on 27 emotion categories and found bots to be more neutral overall, but their messages trigger measurable shifts in human sentiment (more gratitude, admiration, optimism; less confusion).
• Impact inference: Showed that a modest bot presence correlates with both timing and emotional dynamics changes in developer communication.

Methodology

1. Data collection: Scraped all issue and PR comment threads from ten Ethereum‑related GitHub repositories, yielding 36 875 unique accounts.
2. Bot detection & validation: Combined keyword heuristics (e.g., “bot” in usernames), activity signatures (high‑frequency, API‑driven posts), and manual verification to label 105 accounts as bots.
3. Temporal analysis: Computed participation curves (comment counts over the lifespan of an issue/PR) for bots vs. humans, measuring response latency and activity distribution.
4. Emotion classification: Fine‑tuned a transformer‑based model on a labeled dataset covering 27 emotions (e.g., gratitude, confusion, optimism). Each comment was assigned a probability distribution across these categories.
5. Statistical testing: Applied mixed‑effects regression to isolate the effect of a preceding bot comment on the subsequent human comment’s emotional profile, controlling for thread length, repository, and developer experience.

Results & Findings

• Bot prevalence: Bots constitute only 0.28 % of participants but are active in 12 % of all discussion threads.
• Activity curves: Human comments peak early and late (U‑shape), while bots maintain a flat contribution rate on PRs and a delayed surge on issues (often after the initial human discussion).
• Response time: Median bot reply time on PRs is ~2 minutes versus ~15 minutes for humans; on issues, bots reply after a median of 3 hours, compared to 30 minutes for humans.
• Emotional shift: After a bot comment, the probability of human comments expressing gratitude rises by 8 pp, admiration by 5 pp, and optimism by 6 pp, while confusion drops by 7 pp. Overall neutrality drops from 62 % to 48 % in the next human comment.
• Statistical significance: All observed shifts survive Bonferroni‑corrected tests (p < 0.01), indicating a robust association between bot interventions and emotional dynamics.

Practical Implications

• Bot design for morale: Developers building automation (e.g., CI bots, dependency updaters) can intentionally craft messages that foster positive emotions—adding polite phrasing or brief acknowledgments may amplify gratitude and optimism among contributors.
• Workflow optimization: Knowing that bots respond instantly on PRs suggests they are ideal for fast feedback loops (linting, test results). Conversely, their slower issue‑handling pattern implies they should be used for periodic maintenance tasks rather than urgent triage.
• Community health monitoring: Project maintainers can track emotional shifts after bot deployments to gauge community reception, using similar emotion‑classification pipelines as a lightweight sentiment‑dashboard.
• Onboarding assistance: Bots that reduce confusion (e.g., by automatically linking documentation) could accelerate newcomer onboarding, potentially lowering the barrier to entry for complex ecosystems like Ethereum.
• Policy & governance: Organizations can set guidelines for bot tone (e.g., “always include a thank‑you line”) to align automated interactions with the desired culture of the project.

Limitations & Future Work

• Domain specificity: The study focuses exclusively on Ethereum‑related repositories; patterns may differ in other ecosystems (e.g., Rust, Python).
• Bot detection bias: Reliance on username heuristics and manual validation could miss stealthy bots or misclassify highly automated human accounts.
• Emotion model granularity: While 27 categories capture nuance, the classifier’s accuracy varies across low‑frequency emotions, potentially under‑estimating subtle sentiment changes.
• Causality vs. correlation: The analysis shows association, not proof that bots cause emotional shifts; future work could employ A/B testing with controlled bot interventions.
• Long‑term effects: The paper examines immediate post‑bot comment sentiment; longitudinal studies could reveal whether repeated bot exposure leads to lasting community mood changes.

Bottom line: Even a handful of well‑placed bots can reshape the rhythm and emotional climate of open‑source collaboration. By understanding these dynamics, developers and project leaders can harness automation not just for code quality, but also for healthier, more engaging community interactions.

Authors

• Matteo Vaccargiu
• Riccardo Lai
• Maria Ilaria Lunesu
• Andrea Pinna
• Giuseppe Destefanis

Paper Information

• arXiv ID: 2601.11138v1
• Categories: cs.SE
• Published: January 16, 2026
• PDF: Download PDF