Library / Web Scraping & Research
Web Scraping & Research
Scrape Hacker News Who Is Hiring Posts into Airtable
When you run it manually, it fetches Hacker News Who is Hiring comments, extracts job details with OpenAI and saves them to Airtable.
What it does
# HN Who is Hiring Scrape ## Overview Automated workflow: HN Who is Hiring Scrape. This workflow integrates 13 different services: stickyNote, httpRequest, filter, airtable, code. It contains 27 nodes and follows best practices for error handling and security. ## Workflow Details - Total Nodes: 27 - Node Types: 13 - Error Handling: ✅ Implemented - Security: ✅ Hardened (no sensitive data) - Documentation: ✅ Complete ## Node Breakdown - When clicking ‘Test workflow’: manualTrigger - Sticky Note: stickyNote - Split Out: splitOut - Sticky Note1: stickyNote - OpenAI Chat Model: lmChatOpenAi - Structured Output Parser: outputParserStructured - Search for Who is hiring posts: httpRequest - Get relevant data: set - Get latest post: filter - Split out children (jobs): splitOut - ... and 17 more nodes ## Usage Instructions 1. Configure Credentials: Set up all required API keys and credentials 2. Update Variables: Replace any placeholder values with actual data 3. Test Workflow: Run in test mode to verify functionality 4. Deploy: Activate the workflow for production use ## Security Notes - All sensitive data has been removed or replaced with placeholders - Error handling is implemented for reliability - Follow security best practices when configuring credentials ## Troubleshooting - Check error logs if workflow fails - Verify all credentials are properly configured - Ensure all required services are accessible
The steps
- When clicking ‘Test workflow’
- Split Out
- OpenAI Chat Model
- Structured Output Parser
- Search for Who is hiring posts
- Get relevant data
- Get latest post
- Split out children (jobs)
- Trun into structured data
- Extract text
- Clean text
- Limit for testing (optional)
- Write results to airtable
- HI API: Get the individual job post
44 steps in total.
Similar automations
Web Scraping & ResearchScrape Hacker News Who Is Hiring Posts into Airtable15 stepsWeb Scraping & ResearchScrape Hacker News Who Is Hiring Posts with OpenAI15 stepsWeb Scraping & ResearchScrape Hacker News Who Is Hiring Posts into Airtable44 stepsWeb Scraping & ResearchScrape Hacker News Hiring Posts into Airtable with GPT15 steps