Questions About Building a Web Scraping and Automation System in ASP.NET

Source: Dev.to

Question

Is it possible to build a scraping and automation system in ASP.NET using tools like HttpClient, HtmlAgilityPack, or PuppeteerSharp without relying on APIs? And for a final‑year project, would structuring it as a complete automation workflow with data scraping, processing, storage, and scheduling make it more practical?

Are there real‑world examples of similar automation systems, such as those used in the Merchant Cash Advance industry? For example, Cloudsquare’s MCA CRM automates data collection and underwriting.