Skip to content
NEWMarketplace and the trig8n Agent: sell what you build, and get plans and changes that are tested before you see them. See what is new

Library / Web Scraping & Research

Web Scraping & Research

Scrape Hacker News Who Is Hiring Posts into Airtable

When you run it manually, it fetches Hacker News Who is Hiring comments, extracts job details with OpenAI and saves them to Airtable.

What it does

# HN Who is Hiring Scrape ## Overview Automated workflow: HN Who is Hiring Scrape. This workflow integrates 13 different services: stickyNote, httpRequest, filter, airtable, code. It contains 27 nodes and follows best practices for error handling and security. ## Workflow Details - Total Nodes: 27 - Node Types: 13 - Error Handling: ✅ Implemented - Security: ✅ Hardened (no sensitive data) - Documentation: ✅ Complete ## Node Breakdown - When clicking ‘Test workflow’: manualTrigger - Sticky Note: stickyNote - Split Out: splitOut - Sticky Note1: stickyNote - OpenAI Chat Model: lmChatOpenAi - Structured Output Parser: outputParserStructured - Search for Who is hiring posts: httpRequest - Get relevant data: set - Get latest post: filter - Split out children (jobs): splitOut - ... and 17 more nodes ## Usage Instructions 1. Configure Credentials: Set up all required API keys and credentials 2. Update Variables: Replace any placeholder values with actual data 3. Test Workflow: Run in test mode to verify functionality 4. Deploy: Activate the workflow for production use ## Security Notes - All sensitive data has been removed or replaced with placeholders - Error handling is implemented for reliability - Follow security best practices when configuring credentials ## Troubleshooting - Check error logs if workflow fails - Verify all credentials are properly configured - Ensure all required services are accessible

The steps

  1. When clicking ‘Test workflow’
  2. Split Out
  3. OpenAI Chat Model
  4. Structured Output Parser
  5. Search for Who is hiring posts
  6. Get relevant data
  7. Get latest post
  8. Split out children (jobs)
  9. Trun into structured data
  10. Extract text
  11. Clean text
  12. Limit for testing (optional)
  13. Write results to airtable
  14. HI API: Get the individual job post

44 steps in total.

Similar automations