Business Opportunity Radar

Automates scraping and analyzing app, Amazon, Reddit, and Product Hunt reviews to identify and rank SaaS market gaps with validated business ideas.

Hidden Business Opportunity Detector Skill

Overview

This skill builds an automated business intelligence engine that:

1. Apify scrapes App Store reviews, Amazon reviews, Reddit, niche forums, and Product Hunt
2. Claude (OpenClaw) deep-analyzes the raw frustrations, recurring requests, and unmet needs
3. Produces a structured market gap report with validated SaaS ideas, scored by opportunity size

This is how the best indie hackers and founders find their next product — systematically.

🔗 Apify: https://www.apify.com/?fpr=dx06p

---

What This Skill Does

• Scrape App Store & Google Play reviews to find what users hate about existing apps
• Scrape Amazon reviews (1–2 stars) to extract product frustrations at scale
• Mine Reddit niche communities for recurring complaints and feature requests
• Crawl niche forums and communities for unmet needs
• Scrape Product Hunt for emerging tools and gaps in the market
• Feed all raw data into Claude for structured opportunity analysis
• Output a ranked list of business opportunities with validation signals
• Generate SaaS idea briefs with positioning, features, and GTM angle

---

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│           HIDDEN BUSINESS OPPORTUNITY DETECTOR                  │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │  LAYER 1 — DATA MINING (Apify)                          │   │
│  │  App Store │ Google Play │ Amazon │ Reddit │ Forums      │   │
│  │  Product Hunt │ G2 │ Trustpilot │ Indie Hackers          │   │
│  └──────────────────────────┬──────────────────────────────┘   │
│                             │                                   │
│  ┌──────────────────────────▼──────────────────────────────┐   │
│  │  LAYER 2 — OPPORTUNITY ANALYSIS ENGINE (Claude)         │   │
│  │                                                         │   │
│  │  • Frustration Extractor  → what people hate/struggle   │   │
│  │  • Pattern Detector       → recurring complaints        │   │
│  │  • Gap Analyzer           → what nobody is building     │   │
│  │  • Opportunity Scorer     → market size x pain level    │   │
│  │  • SaaS Idea Generator    → concrete product briefs     │   │
│  └──────────────────────────┬──────────────────────────────┘   │
│                             │                                   │
│  ┌──────────────────────────▼──────────────────────────────┐   │
│  │  LAYER 3 — OPPORTUNITY REPORT                           │   │
│  │  Ranked ideas │ Validation signals │ GTM angles          │   │
│  │  JSON export │ Markdown report │ Notion / Slack push     │   │
│  └─────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

---

Step 1 — Get Your API Keys

Apify

1. Sign up at https://www.apify.com/?fpr=dx06p
2. Go to Settings → Integrations
3. Copy your token:

   export APIFY_TOKEN=apify_api_xxxxxxxxxxxxxxxx
   

Claude / OpenClaw

export CLAUDE_API_KEY=sk-ant-xxxxxxxxxxxxxxxx

---

Step 2 — Install Dependencies

npm install apify-client axios node-cron dotenv fs-extra

---

Layer 1 — Multi-Source Data Miner (Apify)

Mine App Store & Google Play Reviews

import ApifyClient from 'apify-client';

const apify = new ApifyClient({ token: process.env.APIFY_TOKEN });

// Define the niche and competitor apps to analyze
const TARGET_NICHE = "project management";
const COMPETITOR_APPS = [
  { name: "Notion",   appStoreId: "1232780281", playStoreId: "notion.id" },
  { name: "Asana",    appStoreId: "489969512",  playStoreId: "com.asana.app" },
  { name: "Trello",   appStoreId: "461504587",  playStoreId: "com.trello" },
  { name: "Monday",   appStoreId: "1298450011", playStoreId: "com.monday.monday" }
];

async function scrapeAppReviews() {
  console.log("📱 Scraping App Store & Play Store reviews...");

  const jobs = COMPETITOR_APPS.map(app =>
    Promise.all([

      // App Store — focus on 1-3 star reviews (the gold mine)
      apify.actor("apify/apple-app-store-scraper").call({
        appIds: [app.appStoreId],
        maxReviews: 100,
        filterStars: [1, 2, 3]
      }).then(run => run.dataset().getData())
        .then(d => d.items.map(r => ({
          source: "app_store",
          appName: app.name,
          rating: r.rating,
          review: r.review,
          title: r.title,
          date: r.date,
          country: r.country
        }))),

      // Google Play Store
      apify.actor("apify/google-play-scraper").call({
        appId: app.playStoreId,
        maxReviews: 100,
        filterScore: [1, 2, 3]
      }).then(run => run.dataset().getData())
        .then(d => d.items.map(r => ({
          source: "google_play",
          appName: app.name,
          rating: r.score,
          review: r.text,
          title: r.title || "",
          date: r.date,
          thumbsUp: r.thumbsUp
        })))

    ]).then(results => results.flat())
  );

  const allReviews = await Promise.all(jobs);
  return allReviews.flat();
}

---

Mine Amazon Reviews (1-3 Stars)

async function scrapeAmazonReviews() {
  console.log("📦 Scraping Amazon negative reviews...");

  // Target products in your niche
  const TARGET_PRODUCTS = [
    "https://www.amazon.com/dp/B08N5WRWNW", // productivity tool example
    "https://www.amazon.com/dp/B09G9HD6PD"
  ];

  const jobs = TARGET_PRODUCTS.map(url =>
    apify.actor("apify/amazon-reviews-scraper").call({
      startUrls: [{ url }],
      maxReviews: 100,
      filterByStar: ["one_star", "two_star", "three_star"]
    }).then(run => run.dataset().getData())
      .then(d => d.items.map(r => ({
        source: "amazon",
        productTitle: r.productTitle,
        rating: r.ratingScore,
        review: r.reviewText,
        title: r.reviewTitle,
        date: r.date,
        helpfulVotes: r.helpfulVotes,
        verifiedPurchase: r.verifiedPurchase
      })))
  );

  const results = await Promise.all(jobs);
  return results.flat();
}

---

Mine Reddit Niche Communities

async function scrapeRedditFrustrations() {
  console.log("💬 Scraping Reddit communities...");

  const SUBREDDITS = [
    "r/Entrepreneur",
    "r/SaaS",
    "r/smallbusiness",
    "r/productivity",
    "r/projectmanagement",
    "r/startups",
    "r/indiehackers"
  ];

  const [posts, searchResults] = await Promise.all([

    // Hot/top posts in subreddits
    apify.actor("apify/reddit-scraper").call({
      startUrls: SUBREDDITS.map(s => ({ url: `https://www.reddit.com/${s}/` })),
      maxPostCount: 30,
      maxComments: 15,
      sort: "top"
    }).then(run => run.dataset().getData()),

    // Search for frustration signals
    apify.actor("apify/reddit-search-scraper").call({
      queries: [
        `${TARGET_NICHE} frustrated wish`,
        `${TARGET_NICHE} hate problem broken`,
        `${TARGET_NICHE} alternative looking for better`,
        `${TARGET_NICHE} feature request need`,
        `${TARGET_NICHE} why is there no tool`
      ],
      maxItems: 50
    }).then(run => run.dataset().getData())

  ]);

  return [
    ...posts.items.map(p => ({
      source: "reddit_post",
      subreddit: p.subreddit,
      title: p.title,
      text: p.selftext,
      score: p.score,
      comments: p.numComments,
      url: p.url
    })),
    ...searchResults.items.map(p => ({
      source: "reddit_search",
      subreddit: p.subreddit,
      title: p.title,
      text: p.selftext,
      score: p.score,
      url: p.url
    }))
  ];
}

---

Mine Product Hunt & G2 Reviews

async function scrapeProductIntelligence() {
  console.log("🚀 Scraping Product Hunt & review platforms...");

  const [productHunt, g2] = await Promise.all([

    // Product Hunt — see what's launching and what comments say
    apify.actor("apify/product-hunt-scraper").call({
      mode: "search",
      searchQuery: TARGET_NICHE,
      maxItems: 30
    }).then(run => run.dataset().getData())
      .then(d => d.items.map(p => ({
        source: "product_hunt",
        name: p.name,
        tagline: p.tagline,
        description: p.description,
        upvotes: p.votesCount,
        comments: p.commentsCount,
        topics: p.topics,
        url: p.url
      }))),

    // G2 reviews for competitor software
    apify.actor("apify/website-content-crawler").call({
      startUrls: [
        { url: `https://www.g2.com/categories/${TARGET_NICHE.replace(/\s+/g, '-')}-software` }
      ],
      maxCrawlingDepth: 1,
      maxRequestsPerCrawl: 10
    }).then(run => run.dataset().getData())
      .then(d => d.items.map(p => ({
        source: "g2",
        text: p.text?.slice(0, 2000),
        url: p.url
      })))

  ]);

  return [...productHunt, ...g2];
}

---

Layer 2 — Opportunity Analysis Engine (Claude)

Frustration Extractor

import axios from 'axios';

const claude = axios.create({
  baseURL: 'https://api.anthropic.com/v1',
  headers: {
    'x-api-key': process.env.CLAUDE_API_KEY,
    'anthropic-version': '2023-06-01',
    'Content-Type': 'application/json'
  }
});

async function extractFrustrations(allData) {
  const prompt = `
You are a world-class product researcher and market analyst.

Analyze this raw data from app reviews, Amazon reviews, Reddit posts, and product listings.
Extract every customer frustration, unmet need, and recurring complaint.

NICHE: ${TARGET_NICHE}

RAW DATA (sample):
${JSON.stringify(allData.slice(0, 30), null, 2)}

Respond ONLY in this JSON format:
{
  "frustrations": [
    {
      "theme": "short label",
      "description": "what users are frustrated about",
      "frequency": "how often this comes up (high/medium/low)",
      "emotionalIntensity": "how angry/upset users are (1-10)",
      "affectedSegment": "who experiences this most",
      "evidenceQuotes": ["direct quote 1", "direct quote 2"],
      "sources": ["app_store", "reddit"]
    }
  ],
  "featureRequests": [
    {
      "request": "what users are explicitly asking for",
      "frequency": "high | medium | low",
      "currentWorkaround": "what users do today instead",
      "evidenceQuotes": ["quote"]
    }
  ],
  "recurringPatterns": [
    "pattern 1 observed across multiple sources",
    "pattern 2"
  ],
  "underservedSegments": [
    {
      "segment": "who is being ignored",
      "unmetNeed": "what they need",
      "currentSolution": "what they use today despite it being bad"
    }
  ]
}
`;

  const { data } = await claude.post('/messages', {
    model: "claude-opus-4-5",
    max_tokens: 3000,
    messages: [{ role: "user", content: prompt }]
  });

  return JSON.parse(data.content[0].text.replace(/```json|```/g, '').trim());
}

---

Market Gap Analyzer & SaaS Idea Generator

async function analyzeMarketGaps(frustrations, productIntel) {
  const prompt = `
You are a serial entrepreneur and SaaS product strategist.

Based on these validated customer frustrations and market intelligence, identify
the highest-potential business opportunities and generate concrete SaaS ideas.

FRUSTRATIONS & PATTERNS:
${JSON.stringify(frustrations, null, 2)}

MARKET INTELLIGENCE (existing products):
${JSON.stringify(productIntel.slice(0, 10), null, 2)}

Respond ONLY in this JSON format:
{
  "marketGaps": [
    {
      "gap": "what is clearly missing from the market",
      "evidenceStrength": "strong | moderate | weak",
      "estimatedMarketSize": "niche | small | medium | large",
      "competitionLevel": "none | low | medium | high",
      "urgency": "nice-to-have | important | critical"
    }
  ],
  "saasIdeas": [
    {
      "rank": 1,
      "name": "working product name",
      "oneLiner": "X for Y — one sentence pitch",
      "problem": "exact problem it solves",
      "targetCustomer": "specific ICP (ideal customer profile)",
      "coreFeatures": ["feature 1", "feature 2", "feature 3"],
      "differentiator": "why this beats existing solutions",
      "monetization": "pricing model (per seat | usage | freemium | etc)",
      "estimatedMRR": "rough MRR potential at 100 customers",
      "validationSignals": ["signal from data that confirms the need"],
      "gtmAngle": "how to acquire first 100 customers",
      "buildComplexity": "low | medium | high",
      "opportunityScore": 8,
      "risksAndChallenges": ["risk 1", "risk 2"]
    }
  ],
  "quickWins": [
    {
      "idea": "simplest possible version of a solution",
      "timeToMVP": "estimated days/weeks to build",
      "validationMethod": "how to validate before building"
    }
  ],
  "topRecommendation": "single best opportunity with 1-paragraph reasoning"
}
`;

  const { data } = await claude.post('/messages', {
    model: "claude-opus-4-5",
    max_tokens: 4000,
    messages: [{ role: "user", content: prompt }]
  });

  return JSON.parse(data.content[0].text.replace(/```json|```/g, '').trim());
}

---

Validation Signal Scorer

async function scoreOpportunities(ideas, rawData) {
  const prompt = `
Score each SaaS idea based on the evidence in the raw data.
Apply the Rob Walling (TinySeed) and Paul Graham opportunity frameworks.

IDEAS TO SCORE:
${JSON.stringify(ideas.saasIdeas, null, 2)}

RAW DATA SIGNALS:
- Total reviews analyzed: ${rawData.length}
- Sources: ${[...new Set(rawData.map(r => r.source))].join(', ')}
- Top frustration themes: ${JSON.stringify(ideas.marketGaps?.slice(0, 5))}

Respond ONLY in this JSON format:
{
  "scoredIdeas": [
    {
      "rank": 1,
      "name": "product name",
      "scores": {
        "painLevel":       { "score": 9, "reasoning": "why" },
        "marketSize":      { "score": 7, "reasoning": "why" },
        "competition":     { "score": 8, "reasoning": "why" },
        "buildability":    { "score": 6, "reasoning": "why" },
        "monetization":    { "score": 8, "reasoning": "why" },
        "founderFit":      { "score": 7, "reasoning": "why" }
      },
      "overallScore": 7.5,
      "verdict": "🔥 Build this | ✅ Worth exploring | ⚠️ Risky | ❌ Skip",
      "nextStep": "concrete first action to validate this idea"
    }
  ],
  "winnerIdea": "name of the single best opportunity",
  "executiveSummary": "2-3 sentence summary of the full analysis"
}
`;

  const { data } = await claude.post('/messages', {
    model: "claude-opus-4-5",
    max_tokens: 2500,
    messages: [{ role: "user", content: prompt }]
  });

  return JSON.parse(data.content[0].text.replace(/```json|```/g, '').trim());
}

---

Layer 3 — Opportunity Report Generator

import { writeFileSync } from 'fs';

function generateMarkdownReport(frustrations, gaps, scored, rawDataCount) {
  const top = scored.scoredIdeas.slice(0, 3);
  const date = new Date().toLocaleDateString('en-US', { year: 'numeric', month: 'long', day: 'numeric' });

  return `# 🎯 Business Opportunity Report
**Niche:** ${TARGET_NICHE} | **Date:** ${date} | **Data Points Analyzed:** ${rawDataCount}

---

## Executive Summary
${scored.executiveSummary}

**🏆 Winner Idea: ${scored.winnerIdea}**

---

## Top Market Gaps Identified

${gaps.marketGaps?.slice(0, 5).map((g, i) =>
  `### ${i + 1}. ${g.gap}
- **Evidence:** ${g.evidenceStrength} | **Market:** ${g.estimatedMarketSize} | **Competition:** ${g.competitionLevel}
- **Urgency:** ${g.urgency}`
).join('\n\n')}

---

## Top 3 SaaS Opportunities

${top.map(idea => `### ${idea.rank}. ${idea.name} — Score: ${idea.overallScore}/10 ${idea.verdict}

${gaps.saasIdeas?.find(i => i.name === idea.name)?.oneLiner || ""}

| Dimension | Score | Notes |
|---|---|---|
| Pain Level | ${idea.scores.painLevel.score}/10 | ${idea.scores.painLevel.reasoning} |
| Market Size | ${idea.scores.marketSize.score}/10 | ${idea.scores.marketSize.reasoning} |
| Competition | ${idea.scores.competition.score}/10 | ${idea.scores.competition.reasoning} |
| Buildability | ${idea.scores.buildability.score}/10 | ${idea.scores.buildability.reasoning} |
| Monetization | ${idea.scores.monetization.score}/10 | ${idea.scores.monetization.reasoning} |

**Next Step:** ${idea.nextStep}`
).join('\n\n---\n\n')}

---

## Top Customer Frustrations

${frustrations.frustrations?.slice(0, 8).map((f, i) =>
  `**${i + 1}. ${f.theme}** (Intensity: ${f.emotionalIntensity}/10 | Frequency: ${f.frequency})
> "${f.evidenceQuotes?.[0] || 'No quote available'}"
${f.description}`
).join('\n\n')}

---

## Quick Wins (Ship in Days)

${gaps.quickWins?.map(q =>
  `- **${q.idea}** | Time to MVP: ${q.timeToMVP} | Validate by: ${q.validationMethod}`
).join('\n')}

---
*Generated by Hidden Business Opportunity Detector • Powered by Apify + Claude*
`;
}

---

Master Orchestrator — Full Pipeline

async function runOpportunityDetector(niche = TARGET_NICHE) {
  console.log(`\n🎯 Opportunity Detector started — ${niche}`);
  console.log(`Timestamp: ${new Date().toISOString()}\n`);

  try {
    // STEP 1 — Mine all data sources in parallel
    console.log("[1/5] Mining data sources...");
    const [appReviews, amazonReviews, redditData, productIntel] = await Promise.all([
      scrapeAppReviews(),
      scrapeAmazonReviews(),
      scrapeRedditFrustrations(),
      scrapeProductIntelligence()
    ]);

    const allData = [...appReviews, ...amazonReviews, ...redditData, ...productIntel];
    console.log(`  ✅ ${allData.length} data points collected`);
    console.log(`     App reviews: ${appReviews.length} | Amazon: ${amazonReviews.length}`);
    console.log(`     Reddit: ${redditData.length} | Product intel: ${productIntel.length}`);

    // STEP 2 — Extract frustrations
    console.log("\n[2/5] Extracting frustrations with Claude...");
    const frustrations = await extractFrustrations(allData);
    console.log(`  ✅ ${frustrations.frustrations?.length} frustration themes identified`);
    console.log(`  ✅ ${frustrations.featureRequests?.length} feature requests found`);

    // STEP 3 — Analyze market gaps and generate SaaS ideas
    console.log("\n[3/5] Analyzing market gaps...");
    const gaps = await analyzeMarketGaps(frustrations, productIntel);
    console.log(`  ✅ ${gaps.marketGaps?.length} gaps identified`);
    console.log(`  ✅ ${gaps.saasIdeas?.length} SaaS ideas generated`);

    // STEP 4 — Score all opportunities
    console.log("\n[4/5] Scoring opportunities...");
    const scored = await scoreOpportunities(gaps, allData);
    console.log(`  ✅ Ideas scored | Winner: ${scored.winnerIdea}`);

    // STEP 5 — Generate report
    console.log("\n[5/5] Generating report...");
    const report = generateMarkdownReport(frustrations, gaps, scored, allData.length);
    writeFileSync(`./opportunity-report-${Date.now()}.md`, report);

    const outputJSON = {
      niche,
      analyzedAt: new Date().toISOString(),
      dataPoints: allData.length,
      frustrationThemes: frustrations.frustrations?.length,
      marketGaps: gaps.marketGaps,
      saasIdeas: scored.scoredIdeas,
      winnerIdea: scored.winnerIdea,
      quickWins: gaps.quickWins,
      executiveSummary: scored.executiveSummary
    };

    writeFileSync(`./opportunity-data-${Date.now()}.json`, JSON.stringify(outputJSON, null, 2));
    console.log("\n✅ Reports saved to disk");

    // Optional: push to Slack
    if (process.env.SLACK_WEBHOOK_URL) {
      await axios.post(process.env.SLACK_WEBHOOK_URL, {
        text: `🎯 *Opportunity Report Ready — ${niche}*\n` +
              `📊 ${allData.length} data points analyzed\n` +
              `🏆 Top idea: *${scored.winnerIdea}*\n` +
              `💬 ${scored.executiveSummary}`
      });
    }

    return outputJSON;

  } catch (err) {
    console.error("Pipeline error:", err.message);
    throw err;
  }
}

// Run immediately
runOpportunityDetector("project management tools");

---

Environment Variables

# .env
APIFY_TOKEN=apify_api_xxxxxxxxxxxxxxxx
CLAUDE_API_KEY=sk-ant-xxxxxxxxxxxxxxxx

# Optional notifications
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/xxx/xxx/xxx
NOTION_API_KEY=secret_xxxxxxxxxxxxxxxx

---

Normalized Opportunity Output Schema

{
  "niche": "project management",
  "analyzedAt": "2025-02-25T10:00:00Z",
  "dataPoints": 380,
  "winnerIdea": "AutoStandup",
  "saasIdeas": [
    {
      "rank": 1,
      "name": "AutoStandup",
      "oneLiner": "Async standups that actually get filled out",
      "overallScore": 8.5,
      "verdict": "🔥 Build this",
      "targetCustomer": "Remote engineering teams 5-50 people",
      "estimatedMRR": "$12,000 at 100 customers ($120/mo per team)",
      "timeToMVP": "3 weeks",
      "nextStep": "Post in r/remotework and r/SaaS — ask if this is a real problem",
      "validationSignals": [
        "47 Reddit posts complaining about standups being ignored",
        "3-star Slack reviews: 'nobody fills them out'"
      ]
    }
  ],
  "quickWins": [
    {
      "idea": "Notion template for async standups",
      "timeToMVP": "2 days",
      "validationMethod": "Post on Gumroad, see if anyone pays $9"
    }
  ]
}

---

Best Practices

• Focus on 1–3 star reviews — that's where the real pain lives
• Scrape at least 200+ reviews per competitor for statistically significant patterns
• Always include a "why is there no tool for X" Reddit search — goldmine for gaps
• Cross-validate: an idea is strong only if the same frustration appears in 3+ sources
• The Quick Wins section is perfect for validation before building — ship a landing page first
• Re-run the pipeline on a new niche weekly to build a pipeline of ideas
• Track which ideas get the most Slack/Notion engagement from your team

---

Error Handling

try {
  const data = await scrapeAppReviews();
  return data;
} catch (error) {
  if (error.statusCode === 401) throw new Error("Invalid Apify token");
  if (error.statusCode === 429) throw new Error("Rate limit — reduce concurrent scrapers");
  if (error.message.includes("actor")) throw new Error("Actor not found — verify actor ID");
  throw error;
}

---

Requirements

• Apify account → https://www.apify.com/?fpr=dx06p
• Claude / OpenClaw API key
• Node.js 18+ with apify-client, axios, node-cron, fs-extra
• Optional: Slack, Notion, or Airtable for team collaboration on the output