I Built a Trend Radar for Developers — Here's How the Scoring Engine Works
Source: Dev.to
The Problem
“Trending” is meaningless without context. A repo with 500 GitHub stars means nothing if nobody’s actually using it. A Product Hunt launch with 200 up‑votes means nothing if it disappears in a week.
I wanted a system that asks: Is this signal showing up across multiple independent sources?
If something is trending on Hacker News and companies are hiring for it and npm downloads are growing and funding is flowing into that sector — that’s not noise. That’s signal.
The Data Pipeline
Every 2 hours a cron job pulls from these sources:
- Hacker News – top stories and comments
- GitHub Trending – repos gaining stars
- Product Hunt – new launches
- Reddit – 4 developer subreddits (threshold: score 65+)
- RemoteOK – job postings
- npm – package download trends (134 packages tracked)
- GitHub Stars Velocity – star‑growth rate for 50 repos
- Hiring Index – 632 companies, 884 active job listings
- Wikipedia – page‑view growth on tech topics
- VC Blogs – 124 posts from investor blogs
- Stack Overflow – unanswered‑question counts by tag
- DOE Research – 346 records from research databases
- Funding Context – sector‑level funding heat maps
- Product Intelligence – engagement & breakout scores for 776 products
- GPT Reports – AI‑generated analysis on ~240 signals
Raw signals are written to snapshot files; the scoring engine then processes everything.
10 Scoring Dimensions
Each signal is scored across 10 dimensions. The raw points add up to a maximum of 135, then are normalized to a 0‑100 scale.
| # | Dimension | Max pts | Description |
|---|---|---|---|
| 1 | Snapshot | 0‑20 | Does the signal exist in the current data pull? |
| 2 | Compare | 0‑15 | How does it compare to signals from previous snapshots? |
| 3 | Replay | 0‑15 | Has this signal appeared consistently over time? |
| 4 | Topics | 0‑15 | Is the keyword growing week‑over‑week? Checks topic history and npm download trends. |
| 5 | Analytics | 0‑15 | Cross‑validation: does the signal appear in multiple independent sources? 3+ validations = 15 pts. |
| 6 | Intelligence | 0‑15 | AI analysis: combines GPT‑generated opportunity scores with product‑intelligence data (engagement + breakout scores across 776 products). |
| 7 | Funding | 0‑10 | Is funding flowing into this signal’s sector? Maps signals to funding heat maps (e.g., DevTools → Technology, AI → AI/ML + Technology). |
| 8 | Early Signals | 0‑10 | Research indicators: Wikipedia page‑view growth, VC blog mentions, Stack Overflow activity, DOE research keywords. Individual keywords are indexed (not full phrases) to catch more matches. |
| 9 | Tracking | 0‑10 | Is this signal in our product index of 1,646 tracked products? |
| 10 | Hiring | 0‑10 | Are companies actively hiring for this? Checks company‑name matches, keyword overlap with job titles, and whether the sector has 10+ companies hiring. |
The Key Insight: Cross‑Validation
The most important dimension is Analytics – the cross‑validation check. A single data source can be gamed (e.g., bots up‑voting an HN post, bought GitHub stars). When a signal shows up on HN and has growing npm downloads and companies are hiring for it and GitHub repos are gaining stars, that convergence is very hard to fake.
That’s what separates 8of8 from tools like Exploding Topics (which mainly rely on Google Trends) or raw scraper dumps.
Score Distribution
After processing ~1,000 raw signals, about 120 pass the qualification filter (≥ 3 dimension passes and score ≥ 25/100).
- Top score: 81/100
- Average score: 43/100
- Most signals cluster: 35‑55 range
Signals scoring 70+ consistently become real trends within weeks.
The Stack
Nothing fancy:
- Python scoring engine (
ultimate_opportunities.py) - Flask frontend
- Gunicorn + nginx on a single Vultr VPS (Ubuntu 24)
- Cron jobs for data collection (every 2 h) and email alerts (daily 8 am UTC)
- Stripe for payments
- Signal‑card images auto‑generated with Pillow
- Auto‑posting to Twitter, Bluesky, Mastodon (3× daily)
Total infrastructure cost: ~ $12 / month for the VPS.
What I Learned
-
More data sources beat better algorithms.
The scoring logic is simple weighted math; value comes from breadth, not clever ML. -
Normalization matters.
Raw scores ranged 40‑80 and looked meaningless. Normalizing to 0‑100 made instant sense to users. -
Free tier has to show value.
My first version hid all signals behind a paywall – nobody converted. Now the free tier shows all 120+ signals with scores and checks visible. Pro ($29/mo) unlocks detailed breakdowns, intelligence reports, and email alerts. -
Signal type matters for communication.
A trending HN article isn’t the same as a trending GitHub repo. I had to teach the auto‑posting system the difference to avoid calling blog posts “tools” (embarrassing lesson learned in public).
Try It
The dashboard is live at 8of8.xyz. The free tier shows everything. I’m looking for beta testers – honest feedback in exchange for free Pro access.
- DM me on X or comment below.
The scoring engine runs on a single Python file. No Kubernetes. No micro‑services. Just cron and conviction.
Built by Jose Marquez Alberti. Powered by ASOF Intelligence.