This looks super useful, but from what i've heard, if you try to do this at any meaningful scale your scrapers will be blocked by Cloudflare and the likes
I used to do a lot of web scraping. Cloudflare is an issue, as are a few Cloudflare competitors, but scraping can still be useful. We had contracts with companies we scraped that allowed us to scrape their sites, specifically so that they didn't need to do any integration work to partner with us. The most anyone had to do on the company side was allowlist us with Cloudflare.
Would recommend web scraping as a "growth hack" in that way, we got a lot of partnerships that we wouldn't otherwise have got.