How to Scrape YouTube Data Without the Official API (No Quota Limits)

Source: Dev.to

Introduction

The YouTube Data API imposes strict quota limits (10 000 units per day, with a single search costing 100 units), which restricts you to roughly 100 searches or 10 000 comments per day. By using YouTube’s internal Innertube API and direct page parsing, you can retrieve the same data without these limits.

Accessing Video Data with the Innertube API

Fetch the video page

// Fetch the video page HTML
const html = await fetch(`https://www.youtube.com/watch?v=${videoId}`)
  .then(r => r.text());

Extract the initial data

// Parse the ytInitialData JSON object from the page source
const ytData = JSON.parse(
  html.match(/ytInitialData\s*=\s*({.*?});/)[1]
);

Locate the comments section

// Navigate to the comments section within the parsed data
const contents = ytData.contents.twoColumnWatchNextResults
  .results.results.contents;

const commentSection = contents.find(
  c => c.itemSectionRenderer?.targetId === "comments-section"
);

The scraper automatically handles pagination, extracting all comments along with:

• Author name and channel
• Comment text
• Like count
• Reply count
• Published date
• Verification status of the author

Retrieving Complete Channel Data

Using the same approach you can obtain:

• Subscriber count
• Total number of videos
• Channel description and external links
• Recent video list with view counts
• Channel creation date

Searching YouTube and Getting Structured Results

When performing a search, the scraper returns structured information for each result:

• Video title, URL, and thumbnail
• View count and publish date
• Channel name
• Duration
• Description snippet

Extracting Video Captions and Subtitles

// Get caption tracks from the player response
const captionMatch = html.match(/"captionTracks":\[(.+?)\]/);
const tracks = JSON.parse(`[${captionMatch[1]}]`);

// Fetch the transcript XML for a specific track
const transcript = await fetch(tracks[0].baseUrl)
  .then(r => r.text());

Features

• Supports multiple languages
• Includes timestamps
• Works with auto‑generated captions
• Handles YouTube Shorts

Practical Applications

• Content research – Analyze which topics generate the most views.
• Sentiment analysis – Process comments at scale to gauge audience reaction.
• Competitor monitoring – Track growth metrics of rival channels.
• AI training data – Build datasets from video transcripts.
• Marketing analytics – Measure engagement across channels.

Tools

All four tools mentioned are available on the Apify Store, with source code and documentation on GitHub. Additional resources include 60+ web‑scraping tools, 15 MCP servers for AI agents, and free market‑research reports.