Library / Web Scraping & Research
Web Scraping & Research
Autonomous AI Crawler for Company Websites with Supabase
When you run it, it takes URLs, OpenAI extracts company names and websites, and Supabase stores the companies it crawls.
What it does
## ⚠️ Note 1. Complete video guide for this workflow is available on my YouTube. 2. Remember to add your credentials and configure nodes. 3. If you like this workflow, please subscribe to my YouTube channel and/or my newsletter. Thank you for your support!
The steps
- Text
- URLs
- OpenAI Chat Model
- JSON Parser
- Map company name and website
- Execute workflow
- Get companies
- Select company name and website
- Set social media array
- Merge all data
- Insert new row
- Convert HTML to Markdown
- Retrieve URLs
- Split out URLs
28 steps in total.
Similar automations
Web Scraping & ResearchCrawl Company Websites Autonomously with OpenAI and Supabase28 stepsLead Generation & EnrichmentExtract Company Names and Websites into Supabase48 stepsLead Generation & EnrichmentExtract Company Names and Websites into Supabase48 stepsEmail AutomationMonitor AI crawler policy signals via HTTP, Data Tables, email and Slack23 steps