finviz-crawler

What to consider before installing: - Review crawl4ai behaviour: the crawler imports and uses the 'crawl4ai' SDK. That library may perform remote requests or forward scraped HTML to a hosted scraping service. If you need scraped content to remain local, inspect crawl4ai's docs/source or run the project after replacing crawl4ai usage with a local-only Playwright solution. - Run install in a controlled environment: the installer runs pip installs and downloads Playwright browsers. Use a Python virtual environment (venv) or container to avoid polluting your system Python and to contain network activity. - Inspect scripts before running: scripts/install.py writes service files and enables a user-level systemd/launchd service and creates files under ~/Downloads/Finviz (or other configured paths). The query script can delete article files and DB rows when removing tickers—back up anything important. - Check defaults and paths: SKILL.md and scripts disagree on default paths (~/workspace/finviz vs ~/Downloads/Finviz). Confirm and set explicit --db / --articles-dir options to ensure data goes where you expect. - If you need stronger privacy control: consider disabling automatic service creation, run the crawler manually on demand, or modify the code to use only local Playwright/browser automation (and confirm no third-party endpoints are contacted). - Network and legal considerations: scraping sites may violate terms of service. The code attempts to respect robots.txt and rate limits, but you should ensure compliance with any sites you scrape. If you want, I can: (1) point to the code locations that call crawl4ai for you to inspect further, (2) suggest a safe set of edits to force local-only crawling, or (3) produce commands to run the installer inside a venv and verify network activity during a test run.